-- Logs begin at Sun 2022-10-30 00:10:48 UTC, end at Sun 2022-10-30 01:26:53 UTC. -- Oct 30 00:47:55.019037 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Oct 30 00:47:55.028124 dv-cinder-master-3-smb devstack@keystone.service[81622]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 30 00:47:55 2022] *** Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: nodename: dv-cinder-master-3-smb Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: machine: x86_64 Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: clock source: unix Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: pcre jit disabled Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: detected number of CPU cores: 4 Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: current working directory: / Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: detected binary path: /usr/bin/uwsgi-core Oct 30 00:47:55.030078 dv-cinder-master-3-smb devstack@keystone.service[81622]: your processes number limit is 47680 Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: your memory page size is 4096 bytes Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: detected max file descriptor number: 2048 Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: lock engine: pthread robust mutexes Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: thunder lock: enabled Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 30 00:47:55.030695 dv-cinder-master-3-smb devstack@keystone.service[81622]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 30 00:47:55.070562 dv-cinder-master-3-smb devstack@keystone.service[81622]: Python main interpreter initialized at 0x556796b7a9c0 Oct 30 00:47:55.070562 dv-cinder-master-3-smb devstack@keystone.service[81622]: python threads support enabled Oct 30 00:47:55.070562 dv-cinder-master-3-smb devstack@keystone.service[81622]: your server socket listen backlog is limited to 100 connections Oct 30 00:47:55.070562 dv-cinder-master-3-smb devstack@keystone.service[81622]: your mercy for graceful operations on workers is 90 seconds Oct 30 00:47:55.070914 dv-cinder-master-3-smb devstack@keystone.service[81622]: mapped 403077 bytes (393 KB) for 2 cores Oct 30 00:47:55.070987 dv-cinder-master-3-smb devstack@keystone.service[81622]: *** Operational MODE: preforking *** Oct 30 00:47:55.071260 dv-cinder-master-3-smb devstack@keystone.service[81622]: *** uWSGI is running in multiple interpreter mode *** Oct 30 00:47:55.071260 dv-cinder-master-3-smb devstack@keystone.service[81622]: spawned uWSGI master process (pid: 81622) Oct 30 00:47:55.071469 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Oct 30 00:47:55.073133 dv-cinder-master-3-smb devstack@keystone.service[81622]: spawned uWSGI worker 1 (pid: 81635, cores: 1) Oct 30 00:47:55.073133 dv-cinder-master-3-smb devstack@keystone.service[81622]: spawned uWSGI worker 2 (pid: 81636, cores: 1) Oct 30 00:47:55.073133 dv-cinder-master-3-smb devstack@keystone.service[81622]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 30 00:47:59.411724 dv-cinder-master-3-smb devstack@keystone.service[81636]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:59.411724 dv-cinder-master-3-smb devstack@keystone.service[81636]: from cryptography.utils import int_from_bytes Oct 30 00:47:59.412402 dv-cinder-master-3-smb devstack@keystone.service[81636]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:59.412402 dv-cinder-master-3-smb devstack@keystone.service[81636]: from cryptography.utils import int_from_bytes Oct 30 00:47:59.413882 dv-cinder-master-3-smb devstack@keystone.service[81635]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:59.413882 dv-cinder-master-3-smb devstack@keystone.service[81635]: from cryptography.utils import int_from_bytes Oct 30 00:47:59.414355 dv-cinder-master-3-smb devstack@keystone.service[81635]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:59.414355 dv-cinder-master-3-smb devstack@keystone.service[81635]: from cryptography.utils import int_from_bytes Oct 30 00:47:59.462737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:47:59.462962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:47:59.463078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] command line args: [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:47:59.463350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:47:59.463528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ================================================================================ {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:47:59.463809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] admin_token = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.464056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] config_dir = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.464167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:47:59.464249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.464365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:47:59.464453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] config_source = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.464558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] command line args: [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:47:59.464643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] debug = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.464776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:47:59.465087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ================================================================================ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:47:59.465281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] admin_token = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] config_dir = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.465899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] config_source = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] debug = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_config_append = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_dir = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.466953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_file = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.467826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_config_append = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_dir = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.468961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.469980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] publish_errors = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.470930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.471947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] publish_errors = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] standard_threads = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] use_journal = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.472973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473049 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] use_json = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] use_stderr = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] use_syslog = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] standard_threads = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.473839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.473931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.474233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] use_journal = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.474395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] use_json = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.474627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] use_stderr = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] use_syslog = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.474814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.474962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:59.475040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.475218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.475305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.475457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.475542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.475730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.482948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.483923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.484975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.485906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.486948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.487967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.488991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.489918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490218 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.490935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.491977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.492952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.493918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.494945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.495927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.496965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.497973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.498698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499426 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.499970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.external = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.password = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] auth.token = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.500985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.external = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.password = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.token = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.501955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.502942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.503943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.504913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.505890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506590 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.506987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.507986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.508985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.509907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510355 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.510936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.511875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.512990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.513928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.514010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.514163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.514250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.514444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.515970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.516977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.517928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.518916 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.519994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.520882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.521841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.522970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.523690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.528950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.529929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.530997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.531996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532609 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.532979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533571 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] role.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] role.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.534970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] role.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] role.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.535860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.536909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537589 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.537900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.538959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.539958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.540895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.541848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542472 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.542945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.543912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.544974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.545117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.545196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.545356 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.545437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.connection = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.546941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.547897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.548958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.549981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.550132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.550213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.550286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:47:59.550358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.550597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.550839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.551083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.551314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.551553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.551812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:59.551985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:47:59.580195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.580426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.580651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.580739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.580951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.581178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.593198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.593352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.621477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.621694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.622558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.622791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.683925 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.684109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.739212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.739444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.739534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.739707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.896883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.897100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.897318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.897539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.897756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.897972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.898065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.898233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.898460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.898683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.898903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.899120 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.921770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.921927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922208 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.922994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.923077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.923212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.923421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.934301 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.942078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.954851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.955969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.956064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.956179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.956262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.956394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:47:59.979453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.979657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.979657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:47:59.979794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:47:59.980010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.980095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.980209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:47:59.980316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:47:59.980483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.980565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.980745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.980828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.981002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.981259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.981341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.981646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.990919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.991033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.996557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:47:59.999262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:47:59.999368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.002195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.002296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.002482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.002590 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.002765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.002856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.003961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.004946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.005071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.005216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.005346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.028410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.028531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.029874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.029973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.030183 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.030267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.034888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.035176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.035458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.036723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.037013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.037298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.041812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.042098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.043920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.044209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.048019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.048340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.048449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.048641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.048733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.048933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.049877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.050055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.050144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.050332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.066173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.066448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.066555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.066729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.070327 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.070605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.073558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.073855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.073948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.074154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.078688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.078949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.081826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.085754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.085858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.085939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.086719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.102506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.108855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.108953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.109035 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.109108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.109182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.109273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.109351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.110181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.111766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.112038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.113417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.113573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.113693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.115123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.115403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.116348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.116633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.117024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.117311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.117591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.118053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.118358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.118669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.118954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.119366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.129539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.129830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.130206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.130488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.130828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.131075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.131312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.131412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.131619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.131912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.132199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.134022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.134022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.134022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.147315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.149238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.150243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.152165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.153170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.154601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.155082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.156323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.156535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.156633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.156887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.158013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.158307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.158717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.163223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.164732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.165060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.166552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.167755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.171836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.171836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.171836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.172107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.172107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.172107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.172107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.176406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.176699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.176981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.178199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.178495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.178778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.184943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.185207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:48:00.189365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.189974 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190467 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.190995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:48:00.222269 dv-cinder-master-3-smb devstack@keystone.service[81636]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556796b7a9c0 pid: 81636 (default app) Oct 30 00:48:00.224163 dv-cinder-master-3-smb devstack@keystone.service[81635]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556796b7a9c0 pid: 81635 (default app) Oct 30 00:48:00.225695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b1b86-4ed7-4945-8ecc-b1c1b7b7ccfa None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:00.225984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b1b86-4ed7-4945-8ecc-b1c1b7b7ccfa None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:00.226254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b1b86-4ed7-4945-8ecc-b1c1b7b7ccfa None None] PATH_INFO: `/v3` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:00.230431 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Sun Oct 30 00:48:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:08.555832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.556631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.556776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.562299 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:08.568457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.569047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.569047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.645001 dv-cinder-master-3-smb devstack@keystone.service[81636]: INFO dbcounter [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] Registered counter for database keystone Oct 30 00:48:08.666994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG oslo_db.sqlalchemy.engines [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81636) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:48:08.674689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writer thread running {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:48:08.860813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG passlib.handlers.bcrypt [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81636) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 30 00:48:08.861205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG passlib.handlers.bcrypt [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81636) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 30 00:48:08.922085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:08.923793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:09.075071 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:48:09.522129 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:09.591177 dv-cinder-master-3-smb devstack@keystone.service[81635]: INFO dbcounter [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] Registered counter for database keystone Oct 30 00:48:09.614354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG oslo_db.sqlalchemy.engines [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81635) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:48:09.621151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writer thread running {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:48:09.745474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f58de-11da-40f9-95db-2b57bf573029 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:09.749791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:09.750414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:09.750695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:09.750969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:09.759787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:09.810942 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 30 00:48:09.811206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15ea7f-e85d-4852-b382-fb5d9bc36b01 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:09.821349 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Sun Oct 30 00:48:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 30 00:48:09.827638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:09.844941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70238b02-b2e9-4443-b6c3-fdf74d724f64 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:09.849337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:09.850448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:09.850731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:09.851011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] PATH_INFO: `/v3/projects/6345a90487264af499b3342614a87ec3` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:09.854831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] RBAC: Authorizing `identity:get_project(project_id=6345a90487264af499b3342614a87ec3)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:09.905290 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 30 00:48:09.905624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc951bd-31ff-4593-b21f-82d305673224 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:09.910290 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:48:09 2022] GET /identity/v3/projects/6345a90487264af499b3342614a87ec3 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:12.076980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24713-f511-47ac-abde-5467667923e9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:12.077509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24713-f511-47ac-abde-5467667923e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:12.077509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24713-f511-47ac-abde-5467667923e9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:12.082287 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:12.088021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:12.088490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:12.088581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:12.108071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:12.109586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:12.117367 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 30 00:48:12.387109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24713-f511-47ac-abde-5467667923e9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:12.399509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24713-f511-47ac-abde-5467667923e9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:12.403856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:12.404503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:12.404790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:12.405068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:12.413087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:12.414886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dcac1-89d8-4e89-97f4-9cb2c14f3bad admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:12.424554 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Sun Oct 30 00:48:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:48:12.430209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:12.440114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0e4a-4a7b-4bf7-bc3c-b5a79b177e5d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:12.444739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:12.445367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:12.445653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:12.445940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] PATH_INFO: `/v3/users/e23837cd4788413c9fb97c7f11ede732` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:12.447247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] RBAC: Authorizing `identity:get_user(user_id=e23837cd4788413c9fb97c7f11ede732)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:12.449112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331591e2-2b34-41bc-9734-e89142e39266 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:12.450648 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Sun Oct 30 00:48:12 2022] GET /identity/v3/users/e23837cd4788413c9fb97c7f11ede732 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:14.684386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cc4a-94f9-4923-b9b9-81094f0d3d60 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.684903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cc4a-94f9-4923-b9b9-81094f0d3d60 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.684991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cc4a-94f9-4923-b9b9-81094f0d3d60 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.686298 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:14.691476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.692019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.692111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.710393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:14.711597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:14.725180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede6bd-32ab-4a6f-80d1-7f023604b4c9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.725464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede6bd-32ab-4a6f-80d1-7f023604b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.725739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede6bd-32ab-4a6f-80d1-7f023604b4c9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.727044 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:14.732520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.732801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.733080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.775411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG passlib.handlers.bcrypt [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81635) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 30 00:48:14.775858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG passlib.handlers.bcrypt [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81635) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 30 00:48:14.797333 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:14.825516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:14.827704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:14.927040 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:15.065671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.090182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.170741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acea94a-7130-4847-91b5-17e1eccc5bf5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.194076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.205684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.206337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.206617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.206895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.218856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:15.227402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:15.228524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:15.229520 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:15.234363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.301784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf5a88-a474-4a3f-ba04-8817b27ca96b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.304161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.321559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.339854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.340200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fa45-4bbc-484c-9805-9220dfae8966 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.340729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.341015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.341293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.342265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.343816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.374460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.375093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.375637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.375945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.379448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:15.379448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.379448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.391015 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:15.396599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.398919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:48:15.398919 dv-cinder-master-3-smb devstack@keystone.service[81635]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:48:15.398919 dv-cinder-master-3-smb devstack@keystone.service[81635]: [parameters: {'id': '59675fb88cfe48ce91c7e65209f75d8b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:48:15.398919 dv-cinder-master-3-smb devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:48:15.417825 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:48:15.418719 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:48:15.420172 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:48:15.421016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:48:15.421924 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:48:15.422726 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'id': '59675fb88cfe48ce91c7e65209f75d8b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:48:15.423536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:15.424316 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.425145 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 8/16] 10.222.0.26 () {60 vars in 1137 bytes} [Sun Oct 30 00:48:15 2022] POST /identity/v3/domains => generated 190 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:48:15.426435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.496298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b5dda-bcc2-47cc-b2d6-9f4f1ed056b5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.527876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfa160-9645-45fa-a71c-fb8769fc5458 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.530008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.530655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.530931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.531207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] PATH_INFO: `/v3/users/e23837cd4788413c9fb97c7f11ede732` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.532386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] RBAC: Authorizing `identity:get_user(user_id=e23837cd4788413c9fb97c7f11ede732)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.533855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.535698 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/users/e23837cd4788413c9fb97c7f11ede732 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:15.540856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.555868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.564085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.564941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.565222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.565492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.583415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.584930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:48:15.601553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:15.603611 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:48:15.604634 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:48:15.604634 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:48:15.604634 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:15.604634 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:15.609339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.632365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.648681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21245e-13df-4304-a179-f4691c33eba0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.686958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.687641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.687928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.688205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.689348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.690809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.692595 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:15.698072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.714878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a63d0-d7fd-425d-9bd7-e3d5b6c78c69 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.753106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.753652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.753932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.754205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.754851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:15.755158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.756576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442e419-8830-4324-ab9b-a94ccc45ba9b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.774050 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:15.779791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.801942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267d4bd-a352-4131-8d2f-17367a0dd41c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.839190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.839920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.840200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.840479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] PATH_INFO: `/v3/domains/default/users/e23837cd4788413c9fb97c7f11ede732/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.843061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e23837cd4788413c9fb97c7f11ede732, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.844560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0110d1-1ecd-48bd-8068-a7ac9cfb9524 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.041354 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 11/21] 10.222.0.26 () {58 vars in 1361 bytes} [Sun Oct 30 00:48:15 2022] PUT /identity/v3/domains/default/users/e23837cd4788413c9fb97c7f11ede732/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:18.508667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c8fa-2975-49b3-8a05-d510d7071032 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.509207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c8fa-2975-49b3-8a05-d510d7071032 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.509207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c8fa-2975-49b3-8a05-d510d7071032 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.510860 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:18.517289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ecbfb-e01b-4d79-84f0-6f2884abda64 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.517778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ecbfb-e01b-4d79-84f0-6f2884abda64 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.519548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ecbfb-e01b-4d79-84f0-6f2884abda64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.545343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8f8ecbfb-e01b-4d79-84f0-6f2884abda64 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:18.546933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8f8ecbfb-e01b-4d79-84f0-6f2884abda64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.637221 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 12/23] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:18.710286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454570-fbc3-4f68-8fba-7e6c34cc09c7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.710630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454570-fbc3-4f68-8fba-7e6c34cc09c7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.710840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454570-fbc3-4f68-8fba-7e6c34cc09c7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.712438 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 12/24] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:18.718691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba5c49-59a8-4b9a-81a7-f69eea6325c8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.718983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba5c49-59a8-4b9a-81a7-f69eea6325c8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.719263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba5c49-59a8-4b9a-81a7-f69eea6325c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.760145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9dba5c49-59a8-4b9a-81a7-f69eea6325c8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:18.761269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9dba5c49-59a8-4b9a-81a7-f69eea6325c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.856239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:18.862158 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 13/25] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:18.967480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832026-ce56-476c-ae2e-f0e607ac3068 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.967827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832026-ce56-476c-ae2e-f0e607ac3068 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.968095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832026-ce56-476c-ae2e-f0e607ac3068 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.969689 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 13/26] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:18.975411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b3a9a-f627-4c2d-9019-15ce8f878c69 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.975967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b3a9a-f627-4c2d-9019-15ce8f878c69 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.976469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b3a9a-f627-4c2d-9019-15ce8f878c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.002817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8b5b3a9a-f627-4c2d-9019-15ce8f878c69 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:19.004424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8b5b3a9a-f627-4c2d-9019-15ce8f878c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:19.087082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832026-ce56-476c-ae2e-f0e607ac3068 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.109130 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 14/27] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:19.165076 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e78edc-5521-40f5-84c0-f51c69b1777d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.165710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e78edc-5521-40f5-84c0-f51c69b1777d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.166198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e78edc-5521-40f5-84c0-f51c69b1777d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.171031 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 15/28] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:19.184850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.185810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.186850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.209263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.221582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:19.222866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:19.248432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832026-ce56-476c-ae2e-f0e607ac3068 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.348969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.349533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.349821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.350100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.350806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.352262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.361933 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 16/29] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:19.362889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.448445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.538706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8068f-3c7a-4e63-b337-cbbb094c605c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.612070 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 14/30] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/roles => generated 219 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:19.613015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.640541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.641221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.641506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.641784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.663669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.665319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:48:19.694275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:19.695209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:48:19.696496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:48:19.696496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:48:19.696496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:19.696496 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 17/31] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:19.703833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.865508 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d70bd4-9d0e-4ac5-b187-3964dc867bca None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.913930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e5d5d-fd0d-45b7-87f0-643c637cb751 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.936960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.943396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.944166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.944463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.944742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.965576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.971209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:19.978868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:19.985639 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 15/32] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:19.986400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.987077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.987077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.987077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.987077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.988077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:19.988077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.990866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.020752 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 18/33] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:20.022052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.104860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1becd2-56cd-4514-8342-792c18d12a11 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.136502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616113a-3a07-463b-9aab-bb3db2d09e9d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.141981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.142607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.142901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.143187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.143901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.145433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.171880 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.172501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.172785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.173061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.173733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:20.174036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.175426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd09e7-d41e-46cb-b44a-272e004b5aa5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.184295 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 19/34] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:20.186860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd210-f153-461e-8e87-c3c8300a728b None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.187319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd210-f153-461e-8e87-c3c8300a728b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.187629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd210-f153-461e-8e87-c3c8300a728b None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.190539 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 20/35] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:20.190539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cd210-f153-461e-8e87-c3c8300a728b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.224765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.240430 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 16/36] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/roles => generated 225 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:20.243781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.329228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.331553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cd210-f153-461e-8e87-c3c8300a728b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.391342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb579c2-c451-4743-851c-d0c4bfa7cc4e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.438440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.439435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.440003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.440469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] PATH_INFO: `/v3/users/e23837cd4788413c9fb97c7f11ede732` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.442322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] RBAC: Authorizing `identity:get_user(user_id=e23837cd4788413c9fb97c7f11ede732)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.444239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a33b6-e69f-4ff7-aaaf-0530338fda16 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.446320 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 17/37] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:20 2022] GET /identity/v3/users/e23837cd4788413c9fb97c7f11ede732 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:20.449580 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.450127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.450408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.475614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:20.476387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.495784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.496416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.496710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.496995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.498197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.499705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921c6ff-af1d-4655-81da-5b138a7e0669 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.558317 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 18/38] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:20.558989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.608210 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 21/39] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/projects => generated 301 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:20.610577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5e8fe-17ea-4372-8282-07199061e7dc None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.611114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5e8fe-17ea-4372-8282-07199061e7dc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.611626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5e8fe-17ea-4372-8282-07199061e7dc None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.614203 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 22/40] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:20.622285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.623459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.624000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.628374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.648721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:20.650019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.683658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683674-1810-4e6d-8def-5ba0d4c74bd7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.726874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.727705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.728062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.728406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.729779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.731497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.734148 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 19/41] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:20.739973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.767478 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 23/42] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:20.835420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.856663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.866329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2270e4-76cc-4632-add6-4928ab4403b7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.896382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.907950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.908489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.908783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.909052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.909790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.domain.id=default, user.id=e23837cd4788413c9fb97c7f11ede732) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:20.910103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.911577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.921222 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 20/43] 10.222.0.26 () {56 vars in 1322 bytes} [Sun Oct 30 00:48:20 2022] GET /identity/v3/role_assignments?user.id=e23837cd4788413c9fb97c7f11ede732&scope.domain.id=default&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 530 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:20.967234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d5cd-23a4-46eb-9d0b-a400b6090184 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.008230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.008870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.009153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.009431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.010125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.011538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.049778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.096789 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 24/44] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/roles => generated 223 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:21.171295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c89d09-34a3-4b61-b340-4fd78987e5c6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.211314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.212021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.212309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.212586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.213760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.215270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09172973-78c2-4d74-96f8-ea85245654b2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.216986 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 21/45] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:21.223006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.307869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:21.332603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059b196-612e-414a-be1c-bf76057391c0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.367671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.368294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.368581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.368863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.369519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.370969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a194614-bbef-47f0-bf98-c2ed541c8081 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.445979 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 25/46] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:48:21 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:23.630492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911620c0-2fc4-49db-a49f-89ebb5a7684b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:23.631045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911620c0-2fc4-49db-a49f-89ebb5a7684b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:23.631045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911620c0-2fc4-49db-a49f-89ebb5a7684b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:23.632407 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 22/47] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:23.637585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:23.638214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:23.638502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:23.656647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:23.657848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:23.724868 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 26/48] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:23.995909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911620c0-2fc4-49db-a49f-89ebb5a7684b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.054467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:24.095132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911620c0-2fc4-49db-a49f-89ebb5a7684b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.128506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.129180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.129467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.129746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.130871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.132348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f05653-28c0-4e3c-869c-31148616fce1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.133917 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 23/49] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:24.139882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.197850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:24.241576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78a385-a58a-4ef0-bf7c-6b320c712ee0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.275857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.276493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.276877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.277226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.278047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.280593 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297eccf-773a-460b-a064-62180e5f683f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.400138 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 27/50] 10.222.0.26 () {60 vars in 1140 bytes} [Sun Oct 30 00:48:24 2022] POST /identity/v3/projects => generated 298 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:26.556474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10b61-bdb3-4730-9487-5b0a32e42198 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.556998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10b61-bdb3-4730-9487-5b0a32e42198 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.556998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10b61-bdb3-4730-9487-5b0a32e42198 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.558341 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 24/51] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:26.564350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.564797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.564945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.582752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:26.583998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:26.649905 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 28/52] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:26.916641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c10b61-bdb3-4730-9487-5b0a32e42198 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.016037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c10b61-bdb3-4730-9487-5b0a32e42198 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.041735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:27.050664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.051309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.051618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.051913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.053095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.054578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6660b-0fb3-493b-b7ee-30c5d91338f9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.056196 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 25/53] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:27.062045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.105468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:27.164220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112da6d-6a50-403a-bc17-a3b55f5e93d3 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.201960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.202165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.202443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.202726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.203440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.204929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9efc7-588b-4d34-b2e3-1a2862b3546c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.301705 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 29/54] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:30.442700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600916-0ab2-439f-bd80-ffc71608adf6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.443757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600916-0ab2-439f-bd80-ffc71608adf6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.444250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600916-0ab2-439f-bd80-ffc71608adf6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.445851 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 26/55] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:30.451642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606337f-950b-41ec-859f-aafee972989c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.452115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606337f-950b-41ec-859f-aafee972989c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.452208 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606337f-950b-41ec-859f-aafee972989c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.481808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5606337f-950b-41ec-859f-aafee972989c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:30.483952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5606337f-950b-41ec-859f-aafee972989c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:30.585180 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 30/56] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:30.869013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f9af-1623-4a9c-b482-76a7204dd208 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.869702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f9af-1623-4a9c-b482-76a7204dd208 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.870195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f9af-1623-4a9c-b482-76a7204dd208 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.872395 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 27/57] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:30.880191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.880746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.881227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.909011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:30.910596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:30.920953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564f9af-1623-4a9c-b482-76a7204dd208 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.941795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.011554 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 31/58] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:31.110726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564f9af-1623-4a9c-b482-76a7204dd208 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.159261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.159864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.160152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.160437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.174932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-1afa7be2-880f-4756-a559-1e88f0264b9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:31.180002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:31.180864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:31.182046 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 28/59] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:48:31.187531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.237189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ec14-c3c2-4843-b113-6ecdb9b688b0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.237507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ec14-c3c2-4843-b113-6ecdb9b688b0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.237800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ec14-c3c2-4843-b113-6ecdb9b688b0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.239135 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 29/60] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:31.249292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.249585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.249868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.280805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:31.282593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:31.330051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da54e0f-be38-4468-bc07-44c0aeb80b5c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.342163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.361151 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 30/61] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:31.361867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.378318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.379774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.380530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.380991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.382140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:31.383022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.384732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807a03-866e-4bac-8170-dcfccc2a45c6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.398574 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 32/62] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:31.400287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531332-bd34-4c5c-baca-4da7a1e50911 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.401090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531332-bd34-4c5c-baca-4da7a1e50911 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.402637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531332-bd34-4c5c-baca-4da7a1e50911 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.410255 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 33/63] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:31.411485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84531332-bd34-4c5c-baca-4da7a1e50911 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.441622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.487774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a628d-aaa3-4786-875f-201477ae4f1c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.525099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.526093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.526601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.527122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.539344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9ee40e41-9cb6-4122-bb0c-4b85d7bee062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:31.545160 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:31.546023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:31.546786 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:31.547786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84531332-bd34-4c5c-baca-4da7a1e50911 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.553891 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 31/64] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:31.553891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf5bd2-5b61-40df-8210-2aa9a5847dc2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.553891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf5bd2-5b61-40df-8210-2aa9a5847dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.553891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf5bd2-5b61-40df-8210-2aa9a5847dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.595568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.596636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.597130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.597562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.615801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.617592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.619565 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 34/65] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:31.620599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.677687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a9bf5bd2-5b61-40df-8210-2aa9a5847dc2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:31.678911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a9bf5bd2-5b61-40df-8210-2aa9a5847dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:31.706195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.721630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.729998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51534cc-222f-4501-8e84-898de89b96ec None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.748307 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 32/66] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:31.756567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350dd4-5f03-4b85-b6aa-8624faa4af1f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.756567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350dd4-5f03-4b85-b6aa-8624faa4af1f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.756567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350dd4-5f03-4b85-b6aa-8624faa4af1f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.756567 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 33/67] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:31.756567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8350dd4-5f03-4b85-b6aa-8624faa4af1f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.784349 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.784981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.785273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.785551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.786233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:31.786543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.787955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.796560 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 35/68] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:31.797575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.866929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8350dd4-5f03-4b85-b6aa-8624faa4af1f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.901346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964693dd-25a9-4be1-a6c6-3b4b79f76c1e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.906727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.907405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.907737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.908030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.909191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.911215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a317b6-dcf0-447f-a46f-cb1c036beafd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.918979 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:31.918979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.918979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.918979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.937689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:31.939020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:31.940752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.941393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.941678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.941942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.949220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.953844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:31.958876 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:31.966275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:31.967032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.986244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.028665 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 35/71] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:32.029506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.082872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6a46-ec96-4d8c-bbf4-dc736c875ea1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.127132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.127809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.128101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.128382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] PATH_INFO: `/v3/users/e23837cd4788413c9fb97c7f11ede732` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.129533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] RBAC: Authorizing `identity:get_user(user_id=e23837cd4788413c9fb97c7f11ede732)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.131025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.132670 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 37/72] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/users/e23837cd4788413c9fb97c7f11ede732 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.133447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.145995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a1759-1beb-4426-83ff-ce4255cf73ef None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.182096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.182689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.182992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.183260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/users/6b908eebe0424e2fb3572a3d75adefe5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.185821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, user_id=6b908eebe0424e2fb3572a3d75adefe5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.187310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.208973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.244189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87c61-633f-457f-b7df-105c2df48da0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.261041 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 36/73] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:32 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/users/6b908eebe0424e2fb3572a3d75adefe5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:32.262138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.305650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.306358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.306644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.306922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.307632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:32.307940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.310295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.310792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.320009 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 38/74] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.320684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.399368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e541b7-af78-4ffe-bf57-beb0bd5bdd8b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.438760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065625f-1de8-4cee-84ea-82cccba3a881 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.443788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.444447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.444731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.445007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.457913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:32.463333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:32.464275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:32.465041 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:32.465724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.476741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.491399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.492340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.492635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.492915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.494131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.495652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.497872 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 39/76] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.498728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.577022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.588521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba9d0f-5989-46e1-836b-ae566e8e249b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.608707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead8b05-effa-46cd-9cc0-1702c0191789 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.625918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.627126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.627427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.627745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.628905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.630813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.632694 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 38/77] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.633412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.649526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.650167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.650448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.650723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.651885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.653944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.659621 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 40/78] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.659621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.743084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.752680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735372f8-09ca-4981-8421-c72a8db378bf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.790205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba2932-d973-4515-b5db-bc5c04a0dbbc None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.793315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.793979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.794265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.794548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.795231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:32.795543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.796969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.816142 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 39/79] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:32.816142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.839235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.840010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.840362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.841414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/users/e23837cd4788413c9fb97c7f11ede732/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.843108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, user_id=e23837cd4788413c9fb97c7f11ede732, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.844213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:32.844614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.924368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b34750-0b86-422d-bc5f-9653768f7dd1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.953087 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 41/80] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:32 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/users/e23837cd4788413c9fb97c7f11ede732/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:32.954024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.979189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.979857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.980143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.980420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.981110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.982651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.082846 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 40/81] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:32 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:33.093543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.102877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635436-1ee5-407a-acad-a0fc0b910fa3 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.160095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:33.178475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.179667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.180256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.180755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] PATH_INFO: `/v3/projects/e53de23643d640fc98cb887df51c9535` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.182476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] RBAC: Authorizing `identity:get_project(project_id=e53de23643d640fc98cb887df51c9535)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.185072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.188440 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 42/82] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:32 2022] GET /identity/v3/projects/e53de23643d640fc98cb887df51c9535 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:33.195889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.267150 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:33.326702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c2c21-dac0-4f1f-8a26-82c993f9fa90 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.374563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.375147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.375432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.375743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.377779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.380356 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.381376 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 41/83] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:33 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:33.387130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.392933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b4e5c-e787-400c-a844-a687f5395e67 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.441951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.443607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.444122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.445120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] PATH_INFO: `/v3/projects/e53de23643d640fc98cb887df51c9535/users/6b908eebe0424e2fb3572a3d75adefe5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.448144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e53de23643d640fc98cb887df51c9535, user_id=6b908eebe0424e2fb3572a3d75adefe5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.450725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.519097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ff2e0-1c3e-411a-b573-5dbee1befe8c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.537554 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 43/84] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:33 2022] PUT /identity/v3/projects/e53de23643d640fc98cb887df51c9535/users/6b908eebe0424e2fb3572a3d75adefe5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:33.568531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.569278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.569548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.569832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.579428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.579428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb5910-e1c8-45fd-b55a-9dd4bbe93eb7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.579428 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 42/85] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:33 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:33.581311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.649639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:33.708933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0831-71d5-41e1-9a9f-d78a42b690e5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.749393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.750152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.750447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.750734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/users/6b908eebe0424e2fb3572a3d75adefe5/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.753345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, user_id=6b908eebe0424e2fb3572a3d75adefe5, role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.755300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b19422-13c5-4bf9-9822-f1c6bb7617f0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.850016 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 44/86] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:33 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/users/6b908eebe0424e2fb3572a3d75adefe5/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:35.026486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18302623-0cdf-46fe-8ba9-85c4e2fbd004 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.026986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18302623-0cdf-46fe-8ba9-85c4e2fbd004 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.027061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18302623-0cdf-46fe-8ba9-85c4e2fbd004 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.030155 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 43/87] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:35.042452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89964afb-33d2-4844-ae48-bccfaba0449d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.042961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89964afb-33d2-4844-ae48-bccfaba0449d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.043310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89964afb-33d2-4844-ae48-bccfaba0449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.076245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-89964afb-33d2-4844-ae48-bccfaba0449d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:35.077992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-89964afb-33d2-4844-ae48-bccfaba0449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:35.165641 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 45/88] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:35.714323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18302623-0cdf-46fe-8ba9-85c4e2fbd004 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:35.735753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:35.860531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18302623-0cdf-46fe-8ba9-85c4e2fbd004 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:35.881419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374e19-61df-4d2c-a960-8f3ca4f1d48e None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.881728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374e19-61df-4d2c-a960-8f3ca4f1d48e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.881984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374e19-61df-4d2c-a960-8f3ca4f1d48e None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.883333 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 46/89] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:35.888527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.888814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.889090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.923971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:35.925127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:35.937802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:35.939121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.939416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.939742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.954412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:35.960405 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:35.961254 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:35.962134 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 44/90] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:35.970025 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:36.040001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:36.129251 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 47/91] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:36.149618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:36.197842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46afb-fd22-422b-a685-a218501b028b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:36.269009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:36.269796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.270083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.270359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.271143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:36.271451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:36.272895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:36.288189 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 45/92] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:36.294941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:36.448073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b134161-2e1e-4f76-8b12-7547a86a4df7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:36.499205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:36.499920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.500270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.500559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.502075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:36.503557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b581cc-eff9-47db-a5f0-e44588215af9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:36.509301 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:36 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:36.515326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:36.657143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35be9d-a1da-42a0-b202-2ea8c7adb48d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35be9d-a1da-42a0-b202-2ea8c7adb48d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.657714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35be9d-a1da-42a0-b202-2ea8c7adb48d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.659248 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 49/94] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:36.672144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.672437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.672720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.680166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b30b-9b5d-49b2-a7aa-143f27fe5c1a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:36.694390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:36.695952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:36.737705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:36.738619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.739115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.739666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.742033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:36.745390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:36.784993 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 50/95] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:36.786078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:36.853192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:36.854958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131a8da-733c-4b3c-937f-8bd3aff7b247 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:36.908552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d7f19-a42f-495f-90bc-b5c99c531db6 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:36.952517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:36.953406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.953835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.954282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.966700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-57e56672-8f78-43b9-a69a-7215c87a45ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:36.972170 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:36.973140 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:36.974028 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:36.976057 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 51/96] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:36.977742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad8d1-3add-43f4-a8e7-2053133292ad None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:36.978180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad8d1-3add-43f4-a8e7-2053133292ad None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:36.978955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad8d1-3add-43f4-a8e7-2053133292ad None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:36.981509 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 52/97] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:36.992381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad8d1-3add-43f4-a8e7-2053133292ad None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.040508 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 46/98] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:37.041795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.046729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.047319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.066214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:37.067703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:37.077350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.116979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.149682 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 47/99] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:37.152001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.241106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad8d1-3add-43f4-a8e7-2053133292ad None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.317913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.317913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.317913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.317913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.318285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:37.318285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.318285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e117f22-2d3c-4762-a8e6-265229cd9215 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.318285 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 53/100] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:37.318501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1edb3-feb6-46e9-a516-7c90b56a5d98 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.318592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e871-fc71-4fbe-be3f-b3a5c400507c None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.318592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e871-fc71-4fbe-be3f-b3a5c400507c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.318592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e871-fc71-4fbe-be3f-b3a5c400507c None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.319217 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 54/101] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:37.319655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234e871-fc71-4fbe-be3f-b3a5c400507c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.367739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.368395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.368683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.368966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.370249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.372375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13d6a1-cf55-48d5-af9d-dd5e2c50158f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.374530 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 48/102] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:37.376367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7a-a015-42bb-973d-e39e07c85cbb None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.376665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7a-a015-42bb-973d-e39e07c85cbb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.376947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7a-a015-42bb-973d-e39e07c85cbb None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.378554 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 49/103] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:37.379297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8bd7a-a015-42bb-973d-e39e07c85cbb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.434448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.441332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.452803 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234e871-fc71-4fbe-be3f-b3a5c400507c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.495954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.497547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.497841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.498130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.499310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.501453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4b38d-07cd-4f2e-88fb-0a8f899c904b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.503068 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 55/104] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:37.505192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4ecc-c871-465a-8be0-c5db7af86d55 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.505479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4ecc-c871-465a-8be0-c5db7af86d55 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.505817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4ecc-c871-465a-8be0-c5db7af86d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.510671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8bd7a-a015-42bb-973d-e39e07c85cbb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.531798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c28b4ecc-c871-465a-8be0-c5db7af86d55 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:37.533026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c28b4ecc-c871-465a-8be0-c5db7af86d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:37.546131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.546764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.547043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.547319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] PATH_INFO: `/v3/users/e23837cd4788413c9fb97c7f11ede732` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.548491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] RBAC: Authorizing `identity:get_user(user_id=e23837cd4788413c9fb97c7f11ede732)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.549931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.551512 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 50/105] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/users/e23837cd4788413c9fb97c7f11ede732 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:37.552305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.613268 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 56/106] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:37.614570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.614922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.615197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.634998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:37.636230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:37.679564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325351c-86bd-41b3-a8b7-36bc2e457608 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.681491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.720033 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 57/107] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:37.721488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.734514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.735398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.735902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.736321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.737221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] RBAC: Adding query filter params (role.id=670880c95def468fa84d3961a35799aa, scope.project.id=65c835132b284cd280d18c69c83e5f94, user.id=6b908eebe0424e2fb3572a3d75adefe5) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:37.738036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.740864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.750662 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 51/108] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/role_assignments?user.id=6b908eebe0424e2fb3572a3d75adefe5&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=670880c95def468fa84d3961a35799aa => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:37.752823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:37.759534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:37.885571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347d46-0cf4-46e6-b5ba-ea40e17bab32 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.891168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f4107-52a3-415f-b741-b1bc893e3a19 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:37.933200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.934718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.935123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.935480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.936531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:37.938451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:37.944047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:37.944717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.945004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.945286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.965558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:37.975106 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:37.983616 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 58/109] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:37.984437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.036890 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 52/110] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:48:37 2022] POST /identity/v3/projects => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:38.037762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.086572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.135236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.173921 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca963c2-4eeb-4049-9859-c1e2ea1fb2a6 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.219654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.220301 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.220588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.220868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.222204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810db2-35f5-420f-9e91-d8f4e3c9d1bf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.277291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.279061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.279532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.279843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.291419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:38.297750 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:38.304200 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 53/111] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:38.304945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.320047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:38.321540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:38.384853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.407378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcbe9-5370-487b-a746-5d11c850d348 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.417279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.437703 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 59/112] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:38.438895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.451311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.451977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.452265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.452542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.453227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:38.453540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:38.454961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:38.464126 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 54/113] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:38.464788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.563707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3493a-e215-42d5-9d45-d133919c888c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.597194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a20cda-afa8-4171-bdca-1c31f1f65c05 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.607901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.608640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.608957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.609200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.621911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:38.631305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:38.632237 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:38.632996 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 60/114] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:38.636426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.653002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.654675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.655147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.655646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.656782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:38.657797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:38.660787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:38.682017 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 55/115] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:38.688019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.688484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.707190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.770316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd099-2e3c-4445-846c-aca9b2277203 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.795311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b09cd-f567-492a-84d6-77b5b1b87202 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.806896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.807551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.807866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.808147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.809868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:38.811350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:38.813044 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 61/116] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:38.813842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.840513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.841279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.841573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.841853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] PATH_INFO: `/v3/users/eb18d3869a0d498e84d3c1dedcf0d638` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:38.858153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] RBAC: Authorizing `identity:get_user(user_id=eb18d3869a0d498e84d3c1dedcf0d638)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:38.859771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:38.866068 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 56/117] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/users/eb18d3869a0d498e84d3c1dedcf0d638 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:38.866068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:38.938748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.941265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb7f0a-c3ad-4908-ae5d-b52e9c6862be None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.951651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.985885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a8-ffa8-440e-aa92-ef6ae189169a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:38.997502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:38.998479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:38.998915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:38.999239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.000047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:39.000403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.002311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.021028 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 62/118] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:39.024011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.045130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.046027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.046452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.046895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.048289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.049944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.051741 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 57/119] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:38 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:39.052686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.178465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486b43-dd4b-4e45-bda0-728d5b339108 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.192469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86cfdb-aca3-46de-9b7b-2681febb606e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.240231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.241272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.241755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.242230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.243537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=65c835132b284cd280d18c69c83e5f94, user.id=e23837cd4788413c9fb97c7f11ede732) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:39.244140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.246563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.247059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.271627 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 63/120] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/role_assignments?user.id=e23837cd4788413c9fb97c7f11ede732&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:39.271837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.271837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.272002 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.272080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.272080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.274060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.277231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.281613 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 58/121] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:39.283537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.338329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.494775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e954e53-a2dd-4838-ad6f-4b535a77f99b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.518879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a558c9-dfa2-4765-91f7-ad758efedf97 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.551184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.552402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.552709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.552967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] PATH_INFO: `/v3/users/6b908eebe0424e2fb3572a3d75adefe5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.555621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] RBAC: Authorizing `identity:get_user(user_id=6b908eebe0424e2fb3572a3d75adefe5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.557124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.559209 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 64/122] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/users/6b908eebe0424e2fb3572a3d75adefe5 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:39.560023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.572381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.573339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.573710 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.574704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.662173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.672908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.674522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.682782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf778-771b-4040-b32b-061779ae4405 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.684686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.724947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.725630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.725919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.726203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/users/eb18d3869a0d498e84d3c1dedcf0d638/roles/eb01e579302d4ebca7da518a429b00c8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.728377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, user_id=eb18d3869a0d498e84d3c1dedcf0d638, role_id=eb01e579302d4ebca7da518a429b00c8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.731285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.783840 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 59/123] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:39.784697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.820499 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 65/124] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:39 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/users/eb18d3869a0d498e84d3c1dedcf0d638/roles/eb01e579302d4ebca7da518a429b00c8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:39.822166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.920551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.930980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266f1f4-ea21-4065-9f8d-236ffd3cd28c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.980314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.981510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.981799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.982083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.990644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1cd5-bc51-4b90-bae2-6b789d9dcae9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.042116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.043136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.044036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.044165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] PATH_INFO: `/v3/projects/e53de23643d640fc98cb887df51c9535` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.046138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] RBAC: Authorizing `identity:get_project(project_id=e53de23643d640fc98cb887df51c9535)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.048968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.051398 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 66/125] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/projects/e53de23643d640fc98cb887df51c9535 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:40.057352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:40.077906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.079601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.104110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:40.187638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:40.195548 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 60/126] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:40.201785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:40.209326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4b2e-e7d1-46f0-b2bc-5a714a37b06e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.257368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.258102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.258458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.258808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.259684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] RBAC: Adding query filter params (role.id=670880c95def468fa84d3961a35799aa, scope.project.id=e53de23643d640fc98cb887df51c9535, user.id=6b908eebe0424e2fb3572a3d75adefe5) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:40.260067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.262160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270f1a-840e-4b27-a610-75683d53b572 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.268506 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 67/127] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:40 2022] GET /identity/v3/role_assignments?user.id=6b908eebe0424e2fb3572a3d75adefe5&scope.project.id=e53de23643d640fc98cb887df51c9535&role.id=670880c95def468fa84d3961a35799aa => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:40.337219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea7f40-2139-49c9-aafb-3d4241015134 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.354177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cc36-463c-401a-8ace-4e93d1713a0c None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.354783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cc36-463c-401a-8ace-4e93d1713a0c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.354783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cc36-463c-401a-8ace-4e93d1713a0c None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.357324 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 68/128] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:40.364901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.365187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.365473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.388163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.388442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:40.389373 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.389677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:40.390046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.390496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.392189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.394067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.396524 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 61/129] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:40 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:40.406103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:40.481064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:40.495819 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 69/130] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:40.521782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f0e28-eb95-4736-a237-bf244957a00a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.560364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.561313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.561774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.562225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.563146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] RBAC: Adding query filter params (role.id=b28dbbda4e65402b8caba7d1305fb822, scope.project.id=65c835132b284cd280d18c69c83e5f94, user.id=6b908eebe0424e2fb3572a3d75adefe5) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:40.563671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.565316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.572430 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 62/131] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:40 2022] GET /identity/v3/role_assignments?user.id=6b908eebe0424e2fb3572a3d75adefe5&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=b28dbbda4e65402b8caba7d1305fb822 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:40.768919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:40.796909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:40.875972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8f12-5a7a-4d8c-a265-1d405aab7e5d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.909250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.909875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.910161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.910437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.911523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.913001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd00c4-787d-4295-855c-49424828dc0c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.914554 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 70/132] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:40.920214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:41.025736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62983-370b-489a-af7c-7fbdd1e5a3a5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:41.051729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:41.060315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:41.060928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:41.061212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:41.061486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:41.062136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:41.063566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52cbd-a945-4242-874b-2d7e94a38a06 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:41.146497 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 63/133] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:40 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:42.064862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b548-a1da-4de7-bdbc-d600018c7f18 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.065431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b548-a1da-4de7-bdbc-d600018c7f18 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.065431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b548-a1da-4de7-bdbc-d600018c7f18 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.066814 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 71/134] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:42.072008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.072408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.072573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.091749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:42.092954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:42.162776 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 64/135] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:42.508390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594b548-a1da-4de7-bdbc-d600018c7f18 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:42.603285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.612998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594b548-a1da-4de7-bdbc-d600018c7f18 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:42.649534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:42.650845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.651128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.651401 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.663135 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:42.668174 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:42.669024 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:42.669789 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 72/136] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:42.675181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:42.780944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd887a3-56bd-4f11-801e-c8c9a4e67978 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:42.795287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.819664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:42.820633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.821125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.821607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.822505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:42.823021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:42.825415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:42.834403 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 65/137] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:42.840345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:42.944375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e683d8-ad2a-4b8d-ab52-7155b246c7c9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:42.980867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:42.981518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.981800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.982078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] PATH_INFO: `/v3/users/eb18d3869a0d498e84d3c1dedcf0d638` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.983241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] RBAC: Authorizing `identity:get_user(user_id=eb18d3869a0d498e84d3c1dedcf0d638)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:42.984731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:42.986330 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 73/138] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:42 2022] GET /identity/v3/users/eb18d3869a0d498e84d3c1dedcf0d638 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:42.992061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:43.101283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918041b9-a85e-4af8-a671-1ff849106569 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:43.138042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:43.138678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.138963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.139245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.187223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:43.210567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:43.218122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.315120 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 66/139] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:43.321142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:43.344631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:43.434438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36bd5-5a1a-4ebd-9ea1-62d285c94e0b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:43.471537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:43.472219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.472505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.472786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.473972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:43.475433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8cdb-0755-4aa7-bf96-2a8399e1720b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.477387 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 74/140] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:43 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:43.484797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:43.520929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1322b2-11a0-49a8-af9f-dcef3a24db67 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.521684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1322b2-11a0-49a8-af9f-dcef3a24db67 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.522185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1322b2-11a0-49a8-af9f-dcef3a24db67 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.523879 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 75/141] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:43.530025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187236bf-7c17-421a-9852-5bc57a1569fc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.530658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187236bf-7c17-421a-9852-5bc57a1569fc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.531134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187236bf-7c17-421a-9852-5bc57a1569fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.550895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-187236bf-7c17-421a-9852-5bc57a1569fc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:43.552259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-187236bf-7c17-421a-9852-5bc57a1569fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.572497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:43.605794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ae8f-c924-44a5-9924-e0774bfe7f6a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:43.639057 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 76/142] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:43.641126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca33cc-3034-45a9-b3fe-427d214061e1 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.641895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca33cc-3034-45a9-b3fe-427d214061e1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.642732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca33cc-3034-45a9-b3fe-427d214061e1 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.645112 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 77/143] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:43.650204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.650495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.650776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.652869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:43.653734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.654320 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.654889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.656003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] RBAC: Adding query filter params (role.id=eb01e579302d4ebca7da518a429b00c8, scope.project.id=65c835132b284cd280d18c69c83e5f94, user.id=eb18d3869a0d498e84d3c1dedcf0d638) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:43.656667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:43.659734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4695219-3dbb-4bd6-b736-c2f3bc1d3cff None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.671425 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 67/144] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:43 2022] GET /identity/v3/role_assignments?user.id=eb18d3869a0d498e84d3c1dedcf0d638&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=eb01e579302d4ebca7da518a429b00c8 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:43.677170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:43.678617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.698414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:43.737308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d90982-febd-4e0c-8f3c-0a6753a3ab59 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.737308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d90982-febd-4e0c-8f3c-0a6753a3ab59 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.737781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d90982-febd-4e0c-8f3c-0a6753a3ab59 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.739126 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 68/145] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:43.746933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.747374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.747813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.769920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:43.770459 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 78/146] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:43.775678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.848393 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 69/147] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:43.908300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.012010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935bbf-37e5-4aea-97ca-7b831e070667 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.047443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.048212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.048500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.048820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.051382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.060270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:44.065280 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:44.066355 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:44.067245 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 79/148] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:44.072127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.085496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.128476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.159337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0428dcc-fc85-4719-b8c9-68479a9cfcbb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.181853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978414-2546-408d-8691-3320aa776ef1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.193600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.194240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.194526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.194808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.205554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:44.210532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:44.211478 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:44.212348 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 70/149] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:44.212990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.218020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.218634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.218913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.219186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.219888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:44.220191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.221627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.234496 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 80/150] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:44.234496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.300243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.311408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a225c6-eb3c-453e-aa0d-85d5166816e7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.331155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde70b5-1f46-41a0-abce-a21cee31d725 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.340206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.345893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.346526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.346805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.347080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.348182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.349645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.351202 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 71/151] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:44.352008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.366025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.366714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.366999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.367270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.367981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:44.368287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.369673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.378541 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 81/152] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:44.379358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.455234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137524-92eb-40a0-9554-f2624fa4feba None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.489113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb343-cdf1-49ec-b67e-e6c8e951b1dd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.489407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.490060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.490348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.490633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] PATH_INFO: `/v3/users/4731de2988d946bda7af3a43f115cd07` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.504633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] RBAC: Authorizing `identity:get_user(user_id=4731de2988d946bda7af3a43f115cd07)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.506098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.507770 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 72/153] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/users/4731de2988d946bda7af3a43f115cd07 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:44.508542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.520103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.524914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.525534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.525819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.526097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.526746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.528247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.563422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.604170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c80f0-f04c-4018-9f01-90f164f38212 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.635638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.636758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.637038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.637309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] PATH_INFO: `/v3/users/4731de2988d946bda7af3a43f115cd07` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.637649 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 82/154] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:44 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:44.638560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.644744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] RBAC: Authorizing `identity:get_user(user_id=4731de2988d946bda7af3a43f115cd07)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.646307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.648314 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 73/155] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/users/4731de2988d946bda7af3a43f115cd07 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:44.654491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.771813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da08f63-b1d2-4459-b385-25d33f212900 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.775369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f5982-191f-4dfe-a80a-cc4a7eb47729 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.799267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.811731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.812795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.813196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.813540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.814676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.816199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.818134 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 74/156] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:44.823232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.824476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.824798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.825083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.825329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.826351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.827986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.829883 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 83/157] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:44.836677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.915897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.957800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8b3-e7b4-4f11-8a2d-5ab5c840eced None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.990512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075a6f-9a66-48c7-862f-bb4507a8c2a4 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:45.005595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:45.006746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:45.006746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:45.008774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/4731de2988d946bda7af3a43f115cd07/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:45.010016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, user_id=4731de2988d946bda7af3a43f115cd07, role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:45.012086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86e72e-e41a-410f-acf2-d9c38da18b8f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:45.047029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:45.048124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:45.048637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:45.049029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/4731de2988d946bda7af3a43f115cd07/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:45.055201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, user_id=4731de2988d946bda7af3a43f115cd07, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:45.055201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f339a-48b4-4913-887b-b010c9b03e22 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:45.101575 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 75/158] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:44 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/4731de2988d946bda7af3a43f115cd07/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:45.127157 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 84/159] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:44 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/4731de2988d946bda7af3a43f115cd07/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:46.984686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7238a-146c-4e29-bbc5-f9c6c1aab983 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.985705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7238a-146c-4e29-bbc5-f9c6c1aab983 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.986190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7238a-146c-4e29-bbc5-f9c6c1aab983 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:46.987872 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 76/160] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:46.994031 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f429-b538-44f2-add3-60b65342c810 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.994534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f429-b538-44f2-add3-60b65342c810 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.994607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f429-b538-44f2-add3-60b65342c810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.013870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-fa37f429-b538-44f2-add3-60b65342c810 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.015063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fa37f429-b538-44f2-add3-60b65342c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.089123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbdd8-e16a-4b76-bc40-7764e83f042c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.089482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbdd8-e16a-4b76-bc40-7764e83f042c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.089765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbdd8-e16a-4b76-bc40-7764e83f042c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.090058 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 85/161] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:47.091234 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 77/162] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:47.096461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ab146-416c-4ce4-a774-34aa2a489ed7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.096790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ab146-416c-4ce4-a774-34aa2a489ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.097030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ab146-416c-4ce4-a774-34aa2a489ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.115823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-583ab146-416c-4ce4-a774-34aa2a489ed7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.117068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-583ab146-416c-4ce4-a774-34aa2a489ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.162853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.188187 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 86/163] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:47.345501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73680-b83f-40fb-ad8b-ccb00d75286b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.345781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73680-b83f-40fb-ad8b-ccb00d75286b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.346057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73680-b83f-40fb-ad8b-ccb00d75286b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.347399 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 78/164] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:47.353137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544456b-0a40-426c-b35c-635a50844848 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.353427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544456b-0a40-426c-b35c-635a50844848 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.353702 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544456b-0a40-426c-b35c-635a50844848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.363474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc73680-b83f-40fb-ad8b-ccb00d75286b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.373738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7544456b-0a40-426c-b35c-635a50844848 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.374951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7544456b-0a40-426c-b35c-635a50844848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.422359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.447722 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 87/165] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:47.449732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fe5a-d04f-4a44-b165-7bb21866d504 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.450024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fe5a-d04f-4a44-b165-7bb21866d504 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.450296 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fe5a-d04f-4a44-b165-7bb21866d504 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.454609 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 88/166] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:47.456822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.460091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.460091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.475956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.477178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.478853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc73680-b83f-40fb-ad8b-ccb00d75286b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.518060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.518690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.518966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.519238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.529363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:47.534575 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:47.535618 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 79/167] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:48:47.536436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.553350 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 89/168] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:47.557580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.605855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.657694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c501-c04a-4c67-8629-c6a7e111549e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.681485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71f236-0904-4794-af03-668b921b8e7d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.685750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.710377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.711178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.711521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.711898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.713350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.716308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.718387 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 80/169] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:47.720330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.721215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.721764 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.722301 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.723451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:47.723999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.724230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.725701 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.735061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 90/170] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:47.735920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.835054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fde048-1269-40b1-8554-a350aa738720 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.842798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.847019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde31b4-6717-4920-976c-ce56dd5c06ba None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.868192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.868727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.869011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.869282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.869977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.872023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.881813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.882448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.882778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.883161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.893307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:47.898582 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:47.899730 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 91/171] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:47.900560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.970850 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 81/172] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:47.970850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.010995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d087ee-270e-46db-83f3-dd5f12972356 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.033539 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.053920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.054581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.054862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.055137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] PATH_INFO: `/v3/users/40440c3546e846e6ae928d88d70fd3bd` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.071827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] RBAC: Authorizing `identity:get_user(user_id=40440c3546e846e6ae928d88d70fd3bd)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.073400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.076929 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 92/173] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/users/40440c3546e846e6ae928d88d70fd3bd => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.077184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.100559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.116353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8f06-4253-491b-a0c4-f396cd7ca7e3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.158004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.159404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.159717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.160734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.172245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:48.178198 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:48.179103 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:48.179959 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 82/174] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:48.186259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.193014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1ffc-c06d-4f0a-b52b-021620c3608d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.234064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.234684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.234967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.235846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.236043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:48.236363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.238233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.248638 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 93/175] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.250989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.318054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.324586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8704f-e48b-4856-97cf-81daedb442e3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.348970 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.365483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.366144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.366447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.366701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.368947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.369904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.372479 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 83/176] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.373108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.377051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cc67-2c1a-41dd-895d-b48b24abbddd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.417401 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.418136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.418420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.418695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.419374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:48.419723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.421122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.431819 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 94/177] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.432641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.514595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf16-7c67-4a85-b0e9-0e381b054289 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.555126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.555811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.556096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.556376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] PATH_INFO: `/v3/users/4731de2988d946bda7af3a43f115cd07` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.557579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] RBAC: Authorizing `identity:get_user(user_id=4731de2988d946bda7af3a43f115cd07)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.559235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.561460 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 84/178] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/users/4731de2988d946bda7af3a43f115cd07 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.562641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.565302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bfb69-7b29-4b97-ba46-50ccef163f5f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.598668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.605494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.606614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.607092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.607654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/40440c3546e846e6ae928d88d70fd3bd/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.610125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, user_id=40440c3546e846e6ae928d88d70fd3bd, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.617132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.676291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa072-2f03-4383-bebb-95eda3fb5a63 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.696612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.709788 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 95/179] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:48 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/40440c3546e846e6ae928d88d70fd3bd/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:48.710809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.733172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.733888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.734177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.734451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] PATH_INFO: `/v3/users/4731de2988d946bda7af3a43f115cd07` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.737130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] RBAC: Authorizing `identity:get_user(user_id=4731de2988d946bda7af3a43f115cd07)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.738721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.740476 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 85/180] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/users/4731de2988d946bda7af3a43f115cd07 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:48.747612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.883760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.889354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dae61-1f80-4e18-bbe1-27d6f067a43c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.944747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601c506-1e59-4359-9050-91a55c8975fb None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.948864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.949493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.949807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.950100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.997186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.997881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.998178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.998465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.057821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.060115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.166211 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 86/181] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:48:49.166575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.167872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.172965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.173358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.174869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.283303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463fc6-2844-4fec-ad30-c43d22f9dba2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.352361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.353011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.353298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.353570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.356003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.357475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.359327 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 87/182] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:49.369966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.372520 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 96/183] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:49.382607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.491299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.533522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80111d8f-ea9b-4ab3-b79a-d58c07d3aadf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.543829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723661e-02d5-4ace-acd9-8e86be2f4f48 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.545478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.583885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.584539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.584822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.585164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.586622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.588581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.590792 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 97/184] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:49.598447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.600219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.600837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.601117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.601393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.602124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0, user.id=4731de2988d946bda7af3a43f115cd07) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:49.602434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.612163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9752a-3e31-4017-ad52-6ccac0277b22 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.623414 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 88/185] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/role_assignments?user.id=4731de2988d946bda7af3a43f115cd07&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:49.738347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217866d-815b-4327-91a4-09df69eeb711 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.776247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.777484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.777770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.778046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.778788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] RBAC: Adding query filter params (role.id=b28dbbda4e65402b8caba7d1305fb822, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0, user.id=4731de2988d946bda7af3a43f115cd07) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:49.779096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.780595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ad2eb-ad6b-4c92-91d3-a2f79f8aa369 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.787006 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 98/186] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/role_assignments?user.id=4731de2988d946bda7af3a43f115cd07&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=b28dbbda4e65402b8caba7d1305fb822 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:50.521479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92c94-7b15-4455-a4dd-f372c6975a67 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.522024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92c94-7b15-4455-a4dd-f372c6975a67 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.522024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92c94-7b15-4455-a4dd-f372c6975a67 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.523567 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 89/187] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:50.528801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.529223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.529371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.530649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301a114-bd2f-45ec-9252-e84c218c7d28 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.530929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301a114-bd2f-45ec-9252-e84c218c7d28 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.531202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301a114-bd2f-45ec-9252-e84c218c7d28 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.532641 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 90/188] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:50.537779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.538321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.538694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.550324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:50.551525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:50.558691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:50.560324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:50.578767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.621899 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 99/189] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:50.636164 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 91/190] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:50.890252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.918049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.983876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.993333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c4fc-fc66-4422-a4b4-5643eb91b270 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.028596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771dff-dc8a-40d9-9b3e-82fea182d6d8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.039932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.040904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.041354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.041637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.052415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:51.057198 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:51.058647 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:51.059469 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 100/191] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:51.064568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.069739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.070386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.070673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.070952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.072097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.074116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.079624 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 92/192] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:51.081001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.097578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.175492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458d226-282d-44cc-bdaf-86f410ff57d8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.182416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7652ef-db67-4892-8a7c-2a61706f121d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.208732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.209348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.209627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.209903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.210410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.210564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:51.210868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.212308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c5831-e94b-4db2-9051-561c41c8ad93 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.216663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.217273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.217559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.217834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.218489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.219939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0f4ff-3c1b-470c-8ac1-ec53e397c2b9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.226627 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 101/193] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:51.226627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c319f4-b556-4bb6-ba27-cda707d50b5c None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.226627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c319f4-b556-4bb6-ba27-cda707d50b5c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.226627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c319f4-b556-4bb6-ba27-cda707d50b5c None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.226627 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 102/194] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:51.227106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c319f4-b556-4bb6-ba27-cda707d50b5c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.310710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.326806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c319f4-b556-4bb6-ba27-cda707d50b5c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.359615 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 93/195] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:51 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:51.359615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.359615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.359615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.362636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.363306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.363609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.363908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] PATH_INFO: `/v3/users/a7344ed501d04692a260c47a71bfd3f7` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.366917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:51.368236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:51.379529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] RBAC: Authorizing `identity:get_user(user_id=a7344ed501d04692a260c47a71bfd3f7)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.381064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.388028 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 103/196] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/users/a7344ed501d04692a260c47a71bfd3f7 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:51.389492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.459946 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 94/197] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:51.506734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e283c9-2b93-492d-b605-ecee2d2d5e89 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.543970 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.544583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.544809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.545069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.546203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.547682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.549505 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 104/198] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:51.555042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.665385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84371f4-b6de-4963-90c1-c756c922b7ce None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.686625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.709930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.710624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.710854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.711123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/a7344ed501d04692a260c47a71bfd3f7/roles/eb01e579302d4ebca7da518a429b00c8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.713092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, user_id=a7344ed501d04692a260c47a71bfd3f7, role_id=eb01e579302d4ebca7da518a429b00c8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.715069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.736196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.751050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.799961 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 95/199] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:48:51 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/users/a7344ed501d04692a260c47a71bfd3f7/roles/eb01e579302d4ebca7da518a429b00c8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:51.852173 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33090986-6bcf-41cb-a25d-414ddda9a4e5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.892505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.893165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.893539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.893878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.906830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:51.912336 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:51.913272 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:51.914071 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 105/200] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:48:51.920536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:52.035912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6825b7-15e3-4965-a141-dd65dde8e153 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:52.071287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:52.071943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:52.072225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:52.072504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:52.073696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:52.074015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:52.075458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:52.083865 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 96/201] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:52.090463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:52.165378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:52.197325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ed4a-8bc5-444a-9053-ff108257965b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:52.234304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:52.234949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:52.235226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:52.235505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] PATH_INFO: `/v3/users/40440c3546e846e6ae928d88d70fd3bd` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:52.236668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] RBAC: Authorizing `identity:get_user(user_id=40440c3546e846e6ae928d88d70fd3bd)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:52.238136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:52.239739 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 106/202] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:52 2022] GET /identity/v3/users/40440c3546e846e6ae928d88d70fd3bd => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:52.245494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:52.287937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:52.351425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933ce3e-53c0-4a6d-be52-03f948c1ebfc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:52.388921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:52.389875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:52.390349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:52.390877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:52.469106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:52.470593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:52.516112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:52.577240 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 97/203] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:52.583066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:52.688988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ced5-df63-4328-9452-e0d7535fb027 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:52.724146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:52.725011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:52.725465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:52.725898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:52.727227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:52.728843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57152848-44b8-4b0e-aa0e-828e63f22e18 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:52.730743 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 107/204] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:52 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:52.740714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:52.847621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7ab3-c1b5-4349-b132-0c2659c9d9b0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:52.888075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:52.888780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:52.889063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:52.889342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:52.890081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] RBAC: Adding query filter params (role.id=670880c95def468fa84d3961a35799aa, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0, user.id=40440c3546e846e6ae928d88d70fd3bd) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:52.890394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:52.891855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee41b0-afeb-4228-83bd-5b200d970b06 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:52.897884 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 98/205] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:52 2022] GET /identity/v3/role_assignments?user.id=40440c3546e846e6ae928d88d70fd3bd&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=670880c95def468fa84d3961a35799aa => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:53.730598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deb8b4-a482-479e-9df3-e81bb2669bb6 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:53.731107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deb8b4-a482-479e-9df3-e81bb2669bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:53.731181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deb8b4-a482-479e-9df3-e81bb2669bb6 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:53.732706 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 108/206] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:53.738117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715adfb2-e7c2-4fa0-ab7c-0a830229d86b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:53.738550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715adfb2-e7c2-4fa0-ab7c-0a830229d86b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:53.738653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715adfb2-e7c2-4fa0-ab7c-0a830229d86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:53.745138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:53.757502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-715adfb2-e7c2-4fa0-ab7c-0a830229d86b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:53.758719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-715adfb2-e7c2-4fa0-ab7c-0a830229d86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:53.827364 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 99/207] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:53.831530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed73c7-2faa-439f-803d-633cea547f95 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:53.831850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed73c7-2faa-439f-803d-633cea547f95 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:53.832132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed73c7-2faa-439f-803d-633cea547f95 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:53.833627 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 109/208] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:53.838610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2e288-3c2c-4c56-be4f-2333c7155d46 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:53.839481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2e288-3c2c-4c56-be4f-2333c7155d46 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:53.839791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2e288-3c2c-4c56-be4f-2333c7155d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:53.857835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-70e2e288-3c2c-4c56-be4f-2333c7155d46 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:53.859139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-70e2e288-3c2c-4c56-be4f-2333c7155d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:53.925680 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 100/209] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:54.097240 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed73c7-2faa-439f-803d-633cea547f95 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.106725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.132091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887eec7-06fa-47b7-a1f9-d1f09b8a2cc2 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.132470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887eec7-06fa-47b7-a1f9-d1f09b8a2cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.132685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887eec7-06fa-47b7-a1f9-d1f09b8a2cc2 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.134237 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 101/210] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:54.141146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c323100-463f-442a-831f-22178b21b893 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.141574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c323100-463f-442a-831f-22178b21b893 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.141862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c323100-463f-442a-831f-22178b21b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.162413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1c323100-463f-442a-831f-22178b21b893 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:54.163610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1c323100-463f-442a-831f-22178b21b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:54.205426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed73c7-2faa-439f-803d-633cea547f95 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.221919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.232384 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 102/211] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:54.233281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c323100-463f-442a-831f-22178b21b893 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.241615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.242533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.243131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.243481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.255073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:54.261412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:54.262521 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:54.263557 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 110/212] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:54.268161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.339765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.348782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c323100-463f-442a-831f-22178b21b893 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.373593 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43c2a5-3a69-46e1-bd1b-250a5ebdc88c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.384340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.385166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.385447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.386250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.387360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.388843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.395157 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 103/213] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:54.396010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.414878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.415486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.415799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.416073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.416762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:54.417068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.418441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.427211 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 111/214] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:54.432830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.458935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.501166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b213-d078-40d1-a108-1345afd9c555 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.536678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.537283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.537605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.537887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.538041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297c26-eee2-48c0-96e2-9b47b68820c1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.538538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.540036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.565537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.572887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.573539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.573815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.574084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] PATH_INFO: `/v3/users/a28859e837814991bd9ad37037561712` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.586659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=a28859e837814991bd9ad37037561712)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.588141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.589768 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 112/215] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/users/a28859e837814991bd9ad37037561712 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:54.590529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.630567 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 104/216] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:48:54 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:54.631398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.706340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1426d8-5bc2-4574-bcf3-308f43a98ed7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.750029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.750801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.750975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.751250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.761287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62544636-64b7-4310-b642-25cc5ef2e1ea None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.763010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:54.768137 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:54.769158 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:54.770205 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 113/217] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:54.775288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:54.784626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.804942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] RBAC: Adding query filter params (role.id=670880c95def468fa84d3961a35799aa, user.id=a28859e837814991bd9ad37037561712, scope.system=all) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.812991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.815216 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 105/218] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/role_assignments?user.id=a28859e837814991bd9ad37037561712&scope.system=all&role.id=670880c95def468fa84d3961a35799aa => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:54.829316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:54.896946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fbd77-54ed-4f1f-a000-0a3ee82d0c76 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:54.937260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:54.937986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:54.938270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:54.938546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:54.939242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:54.939551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:54.941025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:54.951879 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 114/219] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:54.957470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:55.078586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16c78-f7ba-4ffe-804a-2e7675ea8af4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:55.114154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:55.114795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:55.115093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:55.115367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] PATH_INFO: `/v3/users/a7344ed501d04692a260c47a71bfd3f7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:55.116534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] RBAC: Authorizing `identity:get_user(user_id=a7344ed501d04692a260c47a71bfd3f7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:55.118054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:55.119676 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 106/220] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:54 2022] GET /identity/v3/users/a7344ed501d04692a260c47a71bfd3f7 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:55.125610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:55.233107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c110dc9-97d9-407d-b730-312f73ddee97 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:55.242712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:55.269891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:55.270521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:55.270795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:55.271128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:55.349685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:55.351095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:55.468235 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 115/221] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:55.474109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:55.538661 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:55.605151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e748-ba44-4d08-b805-05b7d21061d4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:55.640777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:55.641428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:55.641712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:55.641997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:55.643144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:55.644626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041175b8-1f1d-4a8c-9e0c-e48c37b3c4e5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:55.646652 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 107/222] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:48:55 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:55.655984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:55.669591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:55.795771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389a24-2294-4e90-9b49-18870f4f9eb9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:55.842030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:55.843007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:55.843516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:55.844012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:55.844985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] RBAC: Adding query filter params (role.id=eb01e579302d4ebca7da518a429b00c8, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0, user.id=a7344ed501d04692a260c47a71bfd3f7) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:55.845664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:55.847325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0ad39-9278-46ec-9d32-5a3bc0c01268 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:55.856678 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 116/223] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:48:55 2022] GET /identity/v3/role_assignments?user.id=a7344ed501d04692a260c47a71bfd3f7&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=eb01e579302d4ebca7da518a429b00c8 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:57.052688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9399f5b-343a-43c3-a6a6-e938ceb12965 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.053194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9399f5b-343a-43c3-a6a6-e938ceb12965 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.053194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9399f5b-343a-43c3-a6a6-e938ceb12965 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.054530 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 108/224] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:57.060712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f5922-0e7a-4099-971b-9ee8a485c8f9 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.061260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f5922-0e7a-4099-971b-9ee8a485c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.061332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f5922-0e7a-4099-971b-9ee8a485c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.079636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b988a0-8e07-4891-bfb5-1046488d049e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.079636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b988a0-8e07-4891-bfb5-1046488d049e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.079926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b988a0-8e07-4891-bfb5-1046488d049e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.081221 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 109/225] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:57.081322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-629f5922-0e7a-4099-971b-9ee8a485c8f9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:57.082351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-629f5922-0e7a-4099-971b-9ee8a485c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:57.086386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.086673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.086954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.106547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:57.107757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:57.157465 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 117/226] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:48:57.173038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a86d-311e-4a95-b85a-8199f278d3ac None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.173320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a86d-311e-4a95-b85a-8199f278d3ac None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.173738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a86d-311e-4a95-b85a-8199f278d3ac None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.175072 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 118/227] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:48:57.180343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.180627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.180900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.187021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.187689 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 110/228] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:57.199696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:57.205677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:57.270047 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 119/229] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:48:57.430704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.467057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.485993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.540250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe566c1-f679-47db-a78b-fc5711e2c32c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.569307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56f460-03cd-4a2e-8ba3-04914559d9ff None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.573970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.574601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.574877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.575147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.585087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:57.590874 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:57.591780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:48:57.592568 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 111/230] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:48:57.593212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.601807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.606820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.607464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.607782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.608066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.618579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:48:57.623760 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:48:57.624894 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:48:57.625709 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 120/231] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:48:57.626390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.695544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825133d-8974-465b-98e0-8f046d1bb2ca None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.702851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.727610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0da90a-9f9f-496b-a002-13c94ed87d42 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.728810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.729444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.729741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.729990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.731105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:57.732566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:57.734121 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 112/232] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:57.734897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.763063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.763695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.763978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.764250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.764910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:57.765211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:57.766585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:57.774665 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 121/233] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:57.775450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.831294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.834904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b06f4-9185-4a01-8d26-168204b76054 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.867906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.868511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.868791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.869063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.870258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:57.870566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:57.871964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:57.874874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a4b0-ba6c-44cc-a675-aa2ed1790c7d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:57.880064 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 113/234] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:57.880899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:57.910320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:57.910877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:57.911165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:57.911470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] PATH_INFO: `/v3/groups` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:57.912136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:57.913576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:57.917906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:57.978860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5318d4f-0c7e-406b-ab54-70909eed2de5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:58.013139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:58.013767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:58.014051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:58.014331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] PATH_INFO: `/v3/users/715472b52a1e4a03b7f7a734ead2dbe7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:58.027768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] RBAC: Authorizing `identity:get_user(user_id=715472b52a1e4a03b7f7a734ead2dbe7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:58.029280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:58.030859 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 114/235] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:57 2022] GET /identity/v3/users/715472b52a1e4a03b7f7a734ead2dbe7 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:58.031652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:58.051617 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 122/236] 10.222.0.26 () {60 vars in 1134 bytes} [Sun Oct 30 00:48:57 2022] POST /identity/v3/groups => generated 229 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:58.051617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:58.141750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:58.149775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6e52d-1d68-4181-91e2-627ed438e842 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:58.157848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be762e-e090-4df8-af80-49b1ffd6ea11 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:58.175322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:58.193827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:58.194489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:58.194772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:58.195048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] PATH_INFO: `/v3/users/a28859e837814991bd9ad37037561712` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:58.196244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] RBAC: Authorizing `identity:get_user(user_id=a28859e837814991bd9ad37037561712)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:58.197150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:58.197788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:58.197908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:58.198154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:58.198432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:58.199180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] RBAC: Adding query filter params (role.id=eb01e579302d4ebca7da518a429b00c8, user.id=715472b52a1e4a03b7f7a734ead2dbe7, scope.system=all) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:58.199436 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 115/237] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:48:58 2022] GET /identity/v3/users/a28859e837814991bd9ad37037561712 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:58.199538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:58.200979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f006-cea5-49c4-9a6d-772cd0006828 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:58.205265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:58.207167 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 123/238] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Oct 30 00:48:58 2022] GET /identity/v3/role_assignments?user.id=715472b52a1e4a03b7f7a734ead2dbe7&scope.system=all&role.id=eb01e579302d4ebca7da518a429b00c8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:58.318696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affc8-e176-4376-b015-ca8385b6a426 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:58.357169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:58.357886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:58.358164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:58.358436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] PATH_INFO: `/v3/system/users/a28859e837814991bd9ad37037561712/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:58.360004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a28859e837814991bd9ad37037561712, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:58.361484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7d8e9-c524-4e1e-9f7f-a8b725e2dd6e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:58.517320 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 116/239] 10.222.0.26 () {58 vars in 1334 bytes} [Sun Oct 30 00:48:58 2022] PUT /identity/v3/system/users/a28859e837814991bd9ad37037561712/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:00.294503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea32746-164a-4907-a577-025cd2f1633d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.295044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea32746-164a-4907-a577-025cd2f1633d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.295044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea32746-164a-4907-a577-025cd2f1633d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.296615 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 124/240] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:00.301940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ec2d-de18-4225-a8c5-35d3d438446a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.302346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ec2d-de18-4225-a8c5-35d3d438446a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.302527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ec2d-de18-4225-a8c5-35d3d438446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.320770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4cd5ec2d-de18-4225-a8c5-35d3d438446a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:00.321969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4cd5ec2d-de18-4225-a8c5-35d3d438446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:00.345126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:00.363025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62cf48-f1c4-4f53-849b-08a06f2d970f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.363316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62cf48-f1c4-4f53-849b-08a06f2d970f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.363611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62cf48-f1c4-4f53-849b-08a06f2d970f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.365087 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 125/241] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:00.369841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.370133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.370408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.389055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:00.390249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:00.395018 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 117/242] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:00.467725 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 126/243] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:49:00.660428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71c099-3ce8-441c-bc61-55257babe67d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.660819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71c099-3ce8-441c-bc61-55257babe67d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.660993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71c099-3ce8-441c-bc61-55257babe67d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.664662 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 118/244] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:00.665627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:00.670022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28273-12ea-454b-8871-270501a5071c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.670022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28273-12ea-454b-8871-270501a5071c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.670022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28273-12ea-454b-8871-270501a5071c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.690459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fdb28273-12ea-454b-8871-270501a5071c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:00.691845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fdb28273-12ea-454b-8871-270501a5071c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:00.761765 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 119/245] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:00.762683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb28273-12ea-454b-8871-270501a5071c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:00.773420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeb8da-41f4-47a3-ad3e-3771743fe998 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:00.783501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:00.810443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:00.811079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.811360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.811670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.812770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:00.814217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:00.816008 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 127/246] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:00.821560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:00.852854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:00.867351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb28273-12ea-454b-8871-270501a5071c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:00.904104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:00.904741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.905028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.905726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.916832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:00.921419 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:00.922321 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:00.923111 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 120/247] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:00.928216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:00.944047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67026-543c-48fc-9541-5ebae94a26d8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:00.979117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:00.979762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:00.980048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:00.980324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] PATH_INFO: `/v3/groups` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:00.980951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:00.982403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.035250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22486a-0add-437c-95c2-d56609f7753c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.068543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.069189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.069468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.069744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.070404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:01.070708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.072141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.079880 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 121/248] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:49:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:01.080660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.091043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.115949 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 128/249] 10.222.0.26 () {60 vars in 1134 bytes} [Sun Oct 30 00:49:00 2022] POST /identity/v3/groups => generated 226 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:01.125277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.168451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.193455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed6a18-f460-45fa-92a6-b1483391417c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.233216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.233878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.234160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.234498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.246688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:01.248893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e06e7-7dfc-4764-bab3-75148a1e0cd2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:01.252628 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:01.253536 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:01.254318 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 122/250] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:01.259890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.291623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.291983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.292267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.292550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] PATH_INFO: `/v3/users/715472b52a1e4a03b7f7a734ead2dbe7` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.293834 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] RBAC: Authorizing `identity:get_user(user_id=715472b52a1e4a03b7f7a734ead2dbe7)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.295384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.297532 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 129/251] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:49:01 2022] GET /identity/v3/users/715472b52a1e4a03b7f7a734ead2dbe7 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:01.303066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.335687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.372499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500b2e-7610-4e06-8bb4-861bb48b3a59 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.418310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.419279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.419925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.420207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.420896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:01.421307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.422824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.429935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c195c-c0e1-49fd-bdb1-aa324d765709 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.433307 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 123/252] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:01.434838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.439141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.483853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.484527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.485046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.485152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] PATH_INFO: `/v3/system/users/715472b52a1e4a03b7f7a734ead2dbe7/roles/eb01e579302d4ebca7da518a429b00c8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.486537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=715472b52a1e4a03b7f7a734ead2dbe7, role_id=eb01e579302d4ebca7da518a429b00c8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.487997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.551266 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 130/253] 10.222.0.26 () {58 vars in 1334 bytes} [Sun Oct 30 00:49:01 2022] PUT /identity/v3/system/users/715472b52a1e4a03b7f7a734ead2dbe7/roles/eb01e579302d4ebca7da518a429b00c8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:01.561336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51347222-b3ff-4e6c-a52f-81b75b9a7404 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.596783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.604336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.605008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.605290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.605563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] PATH_INFO: `/v3/users/a28859e837814991bd9ad37037561712` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.606837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] RBAC: Authorizing `identity:get_user(user_id=a28859e837814991bd9ad37037561712)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.608417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff21afa-99f9-4e26-a8c8-0f7cad7a1e94 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.610106 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 124/254] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:49:01 2022] GET /identity/v3/users/a28859e837814991bd9ad37037561712 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:01.616278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:01.734444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f368a5-4124-4b72-83f4-91c3ed9cf376 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:01.772523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:01.773153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:01.773437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:01.773715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:01.774526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] RBAC: Adding query filter params (role.id=670880c95def468fa84d3961a35799aa, user.id=a28859e837814991bd9ad37037561712, scope.system=all) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:01.774838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:01.776454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eda49e-eb22-43eb-b4bf-90b29abb4b6e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:01.789272 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 131/255] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Oct 30 00:49:01 2022] GET /identity/v3/role_assignments?user.id=a28859e837814991bd9ad37037561712&scope.system=all&role.id=670880c95def468fa84d3961a35799aa => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:03.826227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877d8e-5965-4080-8c2a-a2edcf539d9f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:03.826740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877d8e-5965-4080-8c2a-a2edcf539d9f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:03.826821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877d8e-5965-4080-8c2a-a2edcf539d9f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:03.828446 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 125/256] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:03.845498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423ca0c-225c-47f0-b455-3da6c3d4f3c8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:03.846003 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423ca0c-225c-47f0-b455-3da6c3d4f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:03.846003 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423ca0c-225c-47f0-b455-3da6c3d4f3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:03.872614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b423ca0c-225c-47f0-b455-3da6c3d4f3c8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:03.873881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b423ca0c-225c-47f0-b455-3da6c3d4f3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:03.902674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:03.946585 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 132/257] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:49:04.167279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e027-80c3-40a3-85e6-c7e38940ceab None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.168042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e027-80c3-40a3-85e6-c7e38940ceab None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.168042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e027-80c3-40a3-85e6-c7e38940ceab None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.173027 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 126/258] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:04.178289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51286-a0b2-4862-9eb8-7161a437b908 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.178580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51286-a0b2-4862-9eb8-7161a437b908 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.178862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51286-a0b2-4862-9eb8-7161a437b908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.190917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133bcd-56cf-47ef-a775-d1815480658c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.191252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133bcd-56cf-47ef-a775-d1815480658c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.191478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133bcd-56cf-47ef-a775-d1815480658c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.193071 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 127/259] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:04.203181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6d54-7d37-49a3-9d97-803202b9cb66 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.204435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6d54-7d37-49a3-9d97-803202b9cb66 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.204725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6d54-7d37-49a3-9d97-803202b9cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.228961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b8b51286-a0b2-4862-9eb8-7161a437b908 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:04.230179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b8b51286-a0b2-4862-9eb8-7161a437b908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:04.237738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9e8d6d54-7d37-49a3-9d97-803202b9cb66 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:04.238935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9e8d6d54-7d37-49a3-9d97-803202b9cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:04.256243 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 133/260] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 30 00:49:04.333276 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 128/261] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:49:04.427536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673dfe-ac31-4aff-a287-0ed78d93902f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.427926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673dfe-ac31-4aff-a287-0ed78d93902f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.428137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673dfe-ac31-4aff-a287-0ed78d93902f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.429748 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 134/262] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:04.436187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.436474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.436760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.461026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0673dfe-ac31-4aff-a287-0ed78d93902f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.462681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:04.463946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:04.474033 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 129/263] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:49:04.478311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0673dfe-ac31-4aff-a287-0ed78d93902f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.487429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.488117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.488441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.488715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.504794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:04.512673 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:04.514842 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:04.516799 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 135/264] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:04.520023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.536890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92adf3f-3ffb-478e-83eb-53495b4baec0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.545526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.546152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.548852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.548852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.548852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:04.548852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.549468 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:49:04.549468 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 00:49:04.549468 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 00:49:04.550398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.563972 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 130/265] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:04.569861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.581555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3523-9703-444e-ae0e-ba6b151464d3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.590343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.590980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.591260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.591535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.603445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.605372 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:49:04.605372 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 00:49:04.605372 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 00:49:04.606617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.608664 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 136/266] 10.222.0.26 () {56 vars in 1201 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:04.614325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.629195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e7bd4-66e0-4c8b-a81f-5a6a0a0c6f72 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.637143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.637886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.638169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.638443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.640622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.642773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.643650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.644787 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 131/267] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:04.654965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f551e23-bc6d-4c84-9b8a-e9d416d48e6c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.667399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.674926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.675285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.675571 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.682266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.688395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:04.695448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:04.698218 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:04.700193 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 137/268] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:04.706061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a90f-181f-47d6-b99f-1eb359f4fe4a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.706208 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.716116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.716689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20a23-bc5b-46ab-ba86-786f48484ba1 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.716958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.717112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.717324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.724204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.725065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.725239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, group_id=921988b07cdb4b8eb1278decd88f4cd0, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.725391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.725609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.726278 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:49:04.726278 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 00:49:04.726278 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 00:49:04.726724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:04.726899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.727932 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:49:04.727932 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 00:49:04.727932 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 00:49:04.728563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.728852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.742929 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 138/269] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:04.749334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.765286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b86af-bdb4-4e72-8ec7-3295f28dc93d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.770070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.770732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.771015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.771331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.772526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.774452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.776473 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 139/270] 10.222.0.26 () {56 vars in 1201 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:04.781642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.793178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9eff-b332-441f-90cc-5a874372f91d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.797940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.798580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.798861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.799194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.802311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:04.804479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:04.807198 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 140/271] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:04.808127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.834723 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 132/272] 10.222.0.26 () {58 vars in 1463 bytes} [Sun Oct 30 00:49:04 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:04.835686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:04.954822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b3e0-2868-418b-aff3-cbc388104dbc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.969166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e18a1-4120-4e04-a7f6-0a1b60e2e411 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:04.978022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:04.986537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:04.987618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:04.989034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:04.989556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:04.999435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c835132b284cd280d18c69c83e5f94, group_id=921988b07cdb4b8eb1278decd88f4cd0, role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.003308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.011370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.012387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.014276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.014778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.032693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:05.038366 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:05.039312 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:05.040154 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 141/273] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:05.040808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.053994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82c657-f8df-4e11-b555-d77e87661e34 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.059456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.060963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.060963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.061290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.079762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:05.089086 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 133/274] 10.222.0.26 () {58 vars in 1463 bytes} [Sun Oct 30 00:49:04 2022] PUT /identity/v3/projects/65c835132b284cd280d18c69c83e5f94/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:05.090271 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:05.099069 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:05.100110 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:05.100928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:05.107820 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 142/275] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:05.117478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.135817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9720116-35f2-4451-8a81-719722c810fd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.164019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:05.246421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91793a-2010-42c2-bd78-4c307baf1322 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.265691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.266331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.266620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.266903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.267611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:05.267918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.269581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.294402 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 143/276] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:05.295233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.295998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.296293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.296567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.297263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:05.297611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.299035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.301954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.318666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e07f7-4dec-45e8-982a-5daf4d00c4a6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.321496 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 134/277] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:05.325785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.327298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.327895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.328174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.328448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.335996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.341626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.343391 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 144/278] 10.222.0.26 () {56 vars in 1201 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:05.349651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.370333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc02b1-b83a-4701-8f82-694eca7819b4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.385189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.385848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.386130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.386409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.391140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.393338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.395351 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 145/279] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:05.401432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.421406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7208d1-3cbd-4d25-9579-4ca157974806 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.444955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.445646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.445933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.446211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.452971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, group_id=921988b07cdb4b8eb1278decd88f4cd0, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.454027 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:49:05.454027 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 00:49:05.454027 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 00:49:05.455489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0612725-d1e8-4705-8000-fe18d82ddd95 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.456163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1023eea-3f66-4214-8211-a6edc8d2c9ac admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.457986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:05.494629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.495277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.495614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.495900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] PATH_INFO: `/v3/users/715472b52a1e4a03b7f7a734ead2dbe7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.498735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] RBAC: Authorizing `identity:get_user(user_id=715472b52a1e4a03b7f7a734ead2dbe7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.499349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.501264 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 135/280] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/users/715472b52a1e4a03b7f7a734ead2dbe7 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:05.507122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:05.577765 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 146/281] 10.222.0.26 () {58 vars in 1463 bytes} [Sun Oct 30 00:49:05 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:05.622178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db7270-fbc0-4111-8392-f0fbec9aea88 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:05.659934 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:05.660552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:05.660832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:05.661104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:05.667574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] RBAC: Adding query filter params (role.id=eb01e579302d4ebca7da518a429b00c8, user.id=715472b52a1e4a03b7f7a734ead2dbe7, scope.system=all) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:05.667912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:05.674719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09534a-4d7c-4ac4-bc8e-7c672f0fefa4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:05.688803 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 136/282] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Oct 30 00:49:05 2022] GET /identity/v3/role_assignments?user.id=715472b52a1e4a03b7f7a734ead2dbe7&scope.system=all&role.id=eb01e579302d4ebca7da518a429b00c8 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:06.031070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2802e-097e-4b0b-978a-201a1c7bca8e None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.031578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2802e-097e-4b0b-978a-201a1c7bca8e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.031678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2802e-097e-4b0b-978a-201a1c7bca8e None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.033014 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 147/283] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:06.039756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786def-263d-4720-b15a-0c1819186532 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.040231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786def-263d-4720-b15a-0c1819186532 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.040329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786def-263d-4720-b15a-0c1819186532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.062918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2e786def-263d-4720-b15a-0c1819186532 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:06.064177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2e786def-263d-4720-b15a-0c1819186532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:06.089341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:06.132466 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 137/284] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:49:06.419294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2802e-097e-4b0b-978a-201a1c7bca8e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:06.438382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2802e-097e-4b0b-978a-201a1c7bca8e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:06.449176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:06.449847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.450131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.450408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.466071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:06.470973 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:06.471874 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:06.472658 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:06.476332 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 148/285] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 58 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:06.482714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786def-263d-4720-b15a-0c1819186532 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:06.497711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786def-263d-4720-b15a-0c1819186532 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:06.506343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:06.506979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.507262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.507539 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.508261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:06.508571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:06.510276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:06.523448 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 138/286] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Oct 30 00:49:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:06.529048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:06.539272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af751-c289-4154-9541-b85c31772ff1 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:06.547848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:06.548512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.548799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.549081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.550216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:06.552206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:06.554165 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 149/287] 10.222.0.26 () {56 vars in 1201 bytes} [Sun Oct 30 00:49:06 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:06.560404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:06.570493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fd8c-956c-4d8c-9d7a-03cb2833f5b7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:06.575634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:06.576272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.576554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.576836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.577986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:06.580189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab858bd-4f72-4ca6-a659-f6b8c6737c79 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:06.582111 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 139/288] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:49:06 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:06.588331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:06.598654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422ccfc-2d86-41b5-afb1-6343f2ae1ef0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:06.603357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:06.604190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.604475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.604759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.607037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=89885cafe3df4086a0cc1cbe93948bd0, group_id=921988b07cdb4b8eb1278decd88f4cd0, role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:06.609941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:06.689127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946e17a-0d71-4d0e-b9f8-3286797122db None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.689470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946e17a-0d71-4d0e-b9f8-3286797122db None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.689678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946e17a-0d71-4d0e-b9f8-3286797122db None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.691109 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 140/289] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:06.696784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:06.697682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:06.707397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:06.707629 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 150/290] 10.222.0.26 () {58 vars in 1463 bytes} [Sun Oct 30 00:49:06 2022] PUT /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0/groups/921988b07cdb4b8eb1278decd88f4cd0/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:06.724137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:06.725504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:06.809379 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 141/291] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:49:07.134206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:07.149260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1b980-0b0f-4f80-a8d4-dee10eed9f8a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:07.157174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:07.157888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:07.158179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:07.158460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:07.174888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:07.179815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:07.182467 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:07.184448 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 151/292] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:07.191818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:07.203430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ae6b-4aed-4d71-af6d-ce9182c72e9f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:07.213673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:07.214357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:07.214650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:07.214929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:07.215637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:07.215951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:07.217713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:07.231538 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 142/293] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:07.237500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:07.249620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7ed41-c6e6-4e2b-88a5-98e6cb1609df admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:07.254194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:07.254824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:07.255104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:07.255379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] PATH_INFO: `/v3/groups/ee10099045d44d19b61f3b62272ebd86` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:07.261745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] RBAC: Authorizing `identity:get_group(group_id=ee10099045d44d19b61f3b62272ebd86)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:07.263704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:07.265467 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 152/294] 10.222.0.26 () {56 vars in 1201 bytes} [Sun Oct 30 00:49:07 2022] GET /identity/v3/groups/ee10099045d44d19b61f3b62272ebd86 => generated 229 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:07.271167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:07.282542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191b33-3375-4d5c-b9a5-e2188f363e1a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:07.287564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:07.288280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:07.288566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:07.288846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] PATH_INFO: `/v3/projects/6345a90487264af499b3342614a87ec3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:07.290618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] RBAC: Authorizing `identity:get_project(project_id=6345a90487264af499b3342614a87ec3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:07.293179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31171ce1-8e44-47a9-b44e-4738d02689a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:07.295175 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 143/295] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Oct 30 00:49:07 2022] GET /identity/v3/projects/6345a90487264af499b3342614a87ec3 => generated 344 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:07.300863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:07.312114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720192-7d44-488d-8048-bac0cb435d6a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:07.316852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:07.317501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:07.317881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:07.318158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] PATH_INFO: `/v3/projects/6345a90487264af499b3342614a87ec3/groups/ee10099045d44d19b61f3b62272ebd86/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:07.324987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6345a90487264af499b3342614a87ec3, group_id=ee10099045d44d19b61f3b62272ebd86, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:07.327765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0087a44-18d8-4365-b1e5-b268ef02f39b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:07.425383 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 153/296] 10.222.0.26 () {58 vars in 1463 bytes} [Sun Oct 30 00:49:07 2022] PUT /identity/v3/projects/6345a90487264af499b3342614a87ec3/groups/ee10099045d44d19b61f3b62272ebd86/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:08.114511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0f237-7c99-462a-a783-c57afa43c7cc None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.188744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0f237-7c99-462a-a783-c57afa43c7cc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.188744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0f237-7c99-462a-a783-c57afa43c7cc None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.188744 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 144/297] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbcb6-e7ed-4d46-a46d-aa4653437be7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbcb6-e7ed-4d46-a46d-aa4653437be7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbcb6-e7ed-4d46-a46d-aa4653437be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c54dbcb6-e7ed-4d46-a46d-aa4653437be7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c54dbcb6-e7ed-4d46-a46d-aa4653437be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:08.189289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:08.193497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf0d9-e3e2-4509-883b-f3cc520878ef None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.195694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf0d9-e3e2-4509-883b-f3cc520878ef None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.195694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf0d9-e3e2-4509-883b-f3cc520878ef None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.198678 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 145/298] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:08.202869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c38e2-524c-4549-8889-656cc3936232 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.203474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c38e2-524c-4549-8889-656cc3936232 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.203987 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c38e2-524c-4549-8889-656cc3936232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.235719 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 154/299] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:08.237932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-550c38e2-524c-4549-8889-656cc3936232 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:08.239163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-550c38e2-524c-4549-8889-656cc3936232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:08.332824 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 146/300] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:49:08.686751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497e2d-5b75-477a-886d-9b3a847e44dc None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.687093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497e2d-5b75-477a-886d-9b3a847e44dc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.687302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497e2d-5b75-477a-886d-9b3a847e44dc None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.688938 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 155/301] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:08.696673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.697000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.697279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.706960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05497e2d-5b75-477a-886d-9b3a847e44dc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:08.725517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:08.726728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:08.760202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:08.863760 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 147/302] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:08.864691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:08.880820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05497e2d-5b75-477a-886d-9b3a847e44dc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:08.922501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:08.923057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:08.923339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:08.923633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:08.936606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:08.942596 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:08.944015 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:08.944880 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 156/303] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:08.951300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:08.971895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.036744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26243-6056-4b16-9d6d-3ff4b6a09b2c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.069219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb238-b3fd-412d-a981-787852679e54 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.082577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.083503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.084094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.084376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.098796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:09.105840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:09.106967 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:09.107904 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 148/304] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:09.114468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.118177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.119759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.120276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.120759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.121702 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:09.122226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.124717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.126859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.139842 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 157/305] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:49:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:09.142519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.234432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb17-b46e-4ce2-b56e-5388dc4a0389 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.251960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.273133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098f6c-dec4-4f14-97bb-ec91e88eace5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.291802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.293533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.293819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.294096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.294769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:09.295074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.299713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.317422 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 149/306] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:09.325213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.333632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.334267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.334547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.334821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.412042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.432442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.433995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.446564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f3af2-8134-4539-9571-8d2dda3efa8e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.483547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.484292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.484635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.484890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.526110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.551449 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 158/307] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:09.552476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.560897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.562752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.642745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.688099 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 150/308] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:09.689802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.696341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7c13-85cf-45da-b36e-8d503453718f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.742737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.743304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.743635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.743918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.757987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:09.763507 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:09.764461 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:49:09.765265 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:09.766375 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 159/309] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:09.767635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.820886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.834867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05684b-b614-48da-970a-59ede6c94b73 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.891882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.892533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.893069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.893069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.895065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.896561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.899658 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 151/310] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:09.901069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.912698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df382f0-41e2-44d2-8aa7-f3e119b11648 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.946172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:09.957891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] PATH_INFO: `/v3/projects/65c835132b284cd280d18c69c83e5f94` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=65c835132b284cd280d18c69c83e5f94)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe450b-94b5-4f99-b5d4-d9dfce591cb8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.973081 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 160/311] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/projects/65c835132b284cd280d18c69c83e5f94 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:09.973532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95894-4537-4e6d-ae92-3ef90d8eef9b None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.973532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95894-4537-4e6d-ae92-3ef90d8eef9b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.973532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95894-4537-4e6d-ae92-3ef90d8eef9b None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.973532 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 161/312] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:09.973532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95894-4537-4e6d-ae92-3ef90d8eef9b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.031181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a27f9-2058-4b83-922c-27411453577d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.078573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.079368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.080602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.080790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:10.081825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.083237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.094403 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 152/313] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:10.095786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.124018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95894-4537-4e6d-ae92-3ef90d8eef9b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.133047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.171055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.171758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.172045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.172664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.173562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.175047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc301a9-acd5-48f2-b622-3cc90c02aaee None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.176998 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 162/314] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:10.178103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.178387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.178666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.204692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:10.205983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:10.234475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.239406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcd19-eedc-416b-85f3-c0de06ca82b8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.286819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.287392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.287702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.287984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.289105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.290611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.292557 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 153/315] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:10.293401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.305170 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 163/316] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:10.310537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.418221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511c39c-8a37-468f-95b8-ae8c0f785777 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.425219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6e7e-3fc7-4a17-9a4e-b72315e1f1f9 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.427563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.467364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.468047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.468333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.468606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.469277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.469397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] RBAC: Adding query filter params (group.id=921988b07cdb4b8eb1278decd88f4cd0, role.id=670880c95def468fa84d3961a35799aa, scope.project.id=65c835132b284cd280d18c69c83e5f94) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:10.469627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.469899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.470179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.470456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.471067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5531b9-8c7a-4e50-a5a8-5e2f8425649a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.478813 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 164/317] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/role_assignments?group.id=921988b07cdb4b8eb1278decd88f4cd0&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=670880c95def468fa84d3961a35799aa => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:10.482992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697c3-eea1-4cb0-99b1-e35331743672 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.482992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697c3-eea1-4cb0-99b1-e35331743672 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.482992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697c3-eea1-4cb0-99b1-e35331743672 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.482992 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 165/318] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:10.482992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b697c3-eea1-4cb0-99b1-e35331743672 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.530404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.575707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.577287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.636602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b697c3-eea1-4cb0-99b1-e35331743672 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.692337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.693120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.693301 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.695483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.695483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] RBAC: Adding query filter params (group.id=921988b07cdb4b8eb1278decd88f4cd0, role.id=b28dbbda4e65402b8caba7d1305fb822, scope.project.id=65c835132b284cd280d18c69c83e5f94) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:10.695483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:10.696168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323e86f-61a8-42ba-a401-3568c7b8b3d9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:10.704034 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 166/319] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/role_assignments?group.id=921988b07cdb4b8eb1278decd88f4cd0&scope.project.id=65c835132b284cd280d18c69c83e5f94&role.id=b28dbbda4e65402b8caba7d1305fb822 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:10.705565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.705856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.706133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.725492 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 154/320] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:10.726592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.730223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:10.731777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:10.773125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.837786 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 167/321] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:10.839509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.853741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:10.887890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165be7-5286-46de-a354-2985bbe1adb3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:10.938679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:10.939325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:10.939622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:10.939932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:10.951699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:10.956659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:10.957557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:10.958991 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 155/322] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:10.964457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:10.983664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad1ab-9873-4093-b951-1dffd268d1be None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.033766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.034680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.034958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.035411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.037161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.039162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.041725 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 168/323] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:11.047487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.085294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.089631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a263312-a763-4799-a9a5-2ded61334a9f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.124415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.133238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.133860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.134180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.134443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.135115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:11.135466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.136899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.147452 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 156/324] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:11.154135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.169033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb15803-1fd0-4c9f-adf5-5e76c8d7fd60 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.215005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.215704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.216512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.216831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.217995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.219505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.221148 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 169/325] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:11.221923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.272025 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c33e2-345d-4a03-88e5-45f2adba72bc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.313972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.314589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.314874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.315152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.331254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.335443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45756e-4aec-4f16-b4c7-6794c79b72b5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.379072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.379849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.379946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.380181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.383546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.395006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.395997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:11.396716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:11.401462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:11.402365 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:11.403167 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 170/326] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:11.404352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.511501 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 157/327] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:11.511501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.518488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8757495-26f0-4abf-afed-e47e6f41a694 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.552612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.553221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.553502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.553775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.554572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] RBAC: Adding query filter params (group.id=921988b07cdb4b8eb1278decd88f4cd0, role.id=670880c95def468fa84d3961a35799aa, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:11.554883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.556387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.562022 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 171/328] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/role_assignments?group.id=921988b07cdb4b8eb1278decd88f4cd0&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=670880c95def468fa84d3961a35799aa => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:11.562022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.591901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.609040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.619610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996079d-3088-452e-8a3d-0eb6c478bf12 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.668235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.668885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.669171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.669450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.670272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:11.670586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.672177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.681743 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 158/329] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:11.687706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.688622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785d60d-9469-4bd0-b149-a3c4dcae460f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.740696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.741442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.741725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.741999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] PATH_INFO: `/v3/projects/89885cafe3df4086a0cc1cbe93948bd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.743233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] RBAC: Authorizing `identity:get_project(project_id=89885cafe3df4086a0cc1cbe93948bd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.744692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.746372 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 172/330] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/projects/89885cafe3df4086a0cc1cbe93948bd0 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:11.757225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.827021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61e695-cc0d-4e2c-a559-f454ee56445f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.866175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.872714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.873335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.873898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.874171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.877337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ee76-dbbb-4327-89b3-4d94d5d2d41e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:11.898467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.922755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:11.923312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:11.923611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:11.923968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] PATH_INFO: `/v3/groups/921988b07cdb4b8eb1278decd88f4cd0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:11.926241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] RBAC: Authorizing `identity:get_group(group_id=921988b07cdb4b8eb1278decd88f4cd0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.927716 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:11.929480 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 173/331] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/groups/921988b07cdb4b8eb1278decd88f4cd0 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:11.936365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:11.956273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:11.957733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:12.042923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fafec-a43f-4d4e-8f4f-535402d48840 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:12.062055 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 159/332] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:49:12.067686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:12.081680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:12.082300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:12.082582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:12.082872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:12.083614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] RBAC: Adding query filter params (group.id=921988b07cdb4b8eb1278decd88f4cd0, role.id=b28dbbda4e65402b8caba7d1305fb822, scope.project.id=89885cafe3df4086a0cc1cbe93948bd0) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:12.083930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:12.085360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:12.091141 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 174/333] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Oct 30 00:49:11 2022] GET /identity/v3/role_assignments?group.id=921988b07cdb4b8eb1278decd88f4cd0&scope.project.id=89885cafe3df4086a0cc1cbe93948bd0&role.id=b28dbbda4e65402b8caba7d1305fb822 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:12.121776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:12.171986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515264d-95ee-47d3-bd21-8b501d882854 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:12.207293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:12.207881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:12.208165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:12.208442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] PATH_INFO: `/v3/projects/6345a90487264af499b3342614a87ec3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:12.209600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] RBAC: Authorizing `identity:get_project(project_id=6345a90487264af499b3342614a87ec3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:12.211079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:12.212832 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 160/334] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:49:12 2022] GET /identity/v3/projects/6345a90487264af499b3342614a87ec3 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:12.220366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:12.247514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:12.325373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8ff5-ea7a-4407-a91e-a090e1c9168e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:12.359268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:12.359906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:12.360184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:12.360458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] PATH_INFO: `/v3/groups/ee10099045d44d19b61f3b62272ebd86` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:12.361570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] RBAC: Authorizing `identity:get_group(group_id=ee10099045d44d19b61f3b62272ebd86)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:12.363014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87e0f7-402c-4399-bf33-e4eec64ed314 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:12.365024 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 175/335] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Oct 30 00:49:12 2022] GET /identity/v3/groups/ee10099045d44d19b61f3b62272ebd86 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:12.371423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:12.485456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851fbd8-d710-4234-a5a0-2869b8195e52 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:12.506450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:12.520487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:12.521098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:12.521378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:12.521653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:12.522378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] RBAC: Adding query filter params (group.id=ee10099045d44d19b61f3b62272ebd86, role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=6345a90487264af499b3342614a87ec3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:12.522688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:12.524164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c57987-e9d8-4fc5-b679-bdf3b48652f6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:12.540026 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 161/336] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Oct 30 00:49:12 2022] GET /identity/v3/role_assignments?group.id=ee10099045d44d19b61f3b62272ebd86&scope.project.id=6345a90487264af499b3342614a87ec3&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:15.164475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1fad-3a5a-4284-98c2-cb5c312df8e7 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:15.165145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1fad-3a5a-4284-98c2-cb5c312df8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:15.165145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1fad-3a5a-4284-98c2-cb5c312df8e7 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:15.166800 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 176/337] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:15.172349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421405-51b2-4d58-ba69-88edbc917ba2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:15.172856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421405-51b2-4d58-ba69-88edbc917ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:15.172939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421405-51b2-4d58-ba69-88edbc917ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:15.201781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-55421405-51b2-4d58-ba69-88edbc917ba2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:15.203337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-55421405-51b2-4d58-ba69-88edbc917ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:15.287364 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 162/338] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:15.323150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c43f-a317-4cca-bf32-fcba64d132bc None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:15.323934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c43f-a317-4cca-bf32-fcba64d132bc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:15.324579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c43f-a317-4cca-bf32-fcba64d132bc None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:15.326956 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 177/339] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:15.333280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:15.333664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:15.333889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:15.356768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:15.357983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:15.460374 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 163/340] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:15.685936 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7c43f-a317-4cca-bf32-fcba64d132bc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:15.780236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:15.811975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:15.824482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7c43f-a317-4cca-bf32-fcba64d132bc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:15.870584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:15.871405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:15.871819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:15.872238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:15.885079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:15.888070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:15.894614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:15.905502 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:15.910826 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:15.911786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:15.911786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:15.911786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:15.911786 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 178/341] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:15.913384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:15.995795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b187a-c992-4c95-9eee-75f02f9bfb42 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.026624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8e6ee-c746-4885-aa43-3bd055f1093a None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.032131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.032732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.033004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.033282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.044584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.046043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.062445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.063059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-56e24592-f528-43c2-b054-24e536cc0789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:16.063183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:16.064053 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:16.064927 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:16.064927 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.064927 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:16.065088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.065088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.065088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:16.065284 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 164/342] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:16.065361 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.065877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee593e8-20f3-4d8e-83fb-884aba6973cb None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.065980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d50d0-02fa-449a-91ac-f5a17707b932 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.066061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee593e8-20f3-4d8e-83fb-884aba6973cb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.066334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee593e8-20f3-4d8e-83fb-884aba6973cb None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.067609 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 165/343] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:16.068636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07353ad-1c3f-4899-aa96-95dff35c4c92 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.068968 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07353ad-1c3f-4899-aa96-95dff35c4c92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.069192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07353ad-1c3f-4899-aa96-95dff35c4c92 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.070495 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 166/344] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:16.071075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07353ad-1c3f-4899-aa96-95dff35c4c92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.091228 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 179/345] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:16.091228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c0dd-4bd3-42f5-aa34-275a6a28a0b6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.091228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c0dd-4bd3-42f5-aa34-275a6a28a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.091228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c0dd-4bd3-42f5-aa34-275a6a28a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.094976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.108143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1b00c0dd-4bd3-42f5-aa34-275a6a28a0b6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:16.110124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1b00c0dd-4bd3-42f5-aa34-275a6a28a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:16.178230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.191037 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 180/346] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:16.191037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.191037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.191037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.196597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07353ad-1c3f-4899-aa96-95dff35c4c92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.206994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:16.208407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:16.229381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.230255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.230536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.231200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.231880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:16.232188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.233625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.249920 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 167/347] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:16.250778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.299618 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 181/348] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:16.300869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.377955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b477e-a9f8-4150-ba70-96fe21378e82 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.382856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.419577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.420282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.420567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.420842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.421490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.423638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.431719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0f792-1aaf-43b7-a996-5e286ae87a3f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.467861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.468476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.468810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.469083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.469729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.471374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.509696 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 168/349] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:16 2022] POST /identity/v3/users => generated 246 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:16.556575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.574478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.583665 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 182/350] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:16 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:16.659795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.703197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742bb9c-1bc2-4bf3-bf5d-cda227a7e17f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.752689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.753354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.753630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.753904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.770326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.770889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:16.794978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:16.796016 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:16.796871 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:16.796871 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.796871 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:16.796871 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 169/351] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:16.803438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.848473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba9b40-1040-420a-b96b-22ce2fa8a263 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.870851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.892010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.892669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.892957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.893236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.896227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.912532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.914887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:16.935275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:16.937293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:16.938177 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:16.938177 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:16.938177 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:16.940133 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 183/352] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:16.948128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.950331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6f25-e2b7-4e2a-9f77-789671bd7c0c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.003153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.004248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.005169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.005665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.007242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:17.007834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.010222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e54a-221b-4736-a808-a19364721c2d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.026888 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 170/353] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:17.032504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412043f-1a70-4078-be4c-61a44fceb392 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.034610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412043f-1a70-4078-be4c-61a44fceb392 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.034976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412043f-1a70-4078-be4c-61a44fceb392 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.037099 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 171/354] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:17.038221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412043f-1a70-4078-be4c-61a44fceb392 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.082988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6a8b-e2e1-405e-be66-6958726000e2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.128746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.129974 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.131377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.131883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.132837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:17.133420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.135572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56670-0cc5-4dec-bab2-1364fee0bf7d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.159736 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 184/355] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:17.159736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.159736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.159736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.176697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412043f-1a70-4078-be4c-61a44fceb392 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.192761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:17.194061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:17.210248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:17.220816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.221434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.221918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.222196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.223172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.224680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.229931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:17.279850 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 185/356] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:17.288493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.327062 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 172/357] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:17 2022] POST /identity/v3/users => generated 249 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:17.414385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac285838-176b-4715-b97a-1ad425fa8ee2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.455647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.456329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.456666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.456949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.457601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.459051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.532747 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 186/358] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:17 2022] POST /identity/v3/users => generated 248 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:17.595683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.717154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783af4-ff41-4a07-8d20-b0ec608375eb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.756500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.757157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.757445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.757724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.758495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.760051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.800323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:49:17.800323 dv-cinder-master-3-smb devstack@keystone.service[81635]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 30 00:49:17.800323 dv-cinder-master-3-smb devstack@keystone.service[81635]: [parameters: {'id': 'a0238b22bedc4507860950b3cb1900c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 30 00:49:17.800323 dv-cinder-master-3-smb devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:49:17.808752 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return ref.to_dict() Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:49:17.809601 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:49:17.810562 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:49:17.811425 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:49:17.812418 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:49:17.813233 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'id': 'a0238b22bedc4507860950b3cb1900c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:17.813999 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 30 00:49:17.814805 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:17.817670 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 173/359] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Oct 30 00:49:17 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:17.817811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.841356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:17.937290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc537-bc0d-48d3-b4b9-0d5b3ac36032 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.987793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.988447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.988735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.989012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.005535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-69834175-d9f1-4086-a25c-647c0b27c6c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:18.011569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:18.013642 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:18.015544 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 187/360] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:18.019016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.096344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:18.169562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b42c-0e3f-4a44-bf25-2a5430737e8c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.214206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.214954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.215109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.215295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.216014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:18.216333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.217729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bcd3c-bdff-491c-b521-2d36380b78c1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.226479 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 174/361] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:19.255436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda1ad-e591-4cf6-855a-43162fbd77e5 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.255989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda1ad-e591-4cf6-855a-43162fbd77e5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.256296 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda1ad-e591-4cf6-855a-43162fbd77e5 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.256526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e7d4-527a-4bf9-969a-7504538f944e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.256892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e7d4-527a-4bf9-969a-7504538f944e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.257090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e7d4-527a-4bf9-969a-7504538f944e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.257847 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 188/362] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:19.258963 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 175/363] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:19.264716 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.265015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.265262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.269087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.269373 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.269649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.301942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:19.303500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:19.326776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:19.326776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:19.413817 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 176/364] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:19.467297 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 189/365] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:19.822735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:19.843818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:19.852869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:19.878870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:19.997100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55503-e33b-47bd-912c-2f8f55b96926 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.020071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63375b-05ce-40bf-85bb-7d110fa7b903 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.053070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.053719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.053997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.054272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.065168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-fe72fd4c-3dbb-4069-a6e5-3aaa1bbe4a32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:20.070275 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:20.072095 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:20.072900 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 190/366] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:20.073927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07627-3edb-4196-ad56-733a67ea9f2b None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.078817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07627-3edb-4196-ad56-733a67ea9f2b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.078817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07627-3edb-4196-ad56-733a67ea9f2b None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.078817 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 191/367] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:20.081411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07627-3edb-4196-ad56-733a67ea9f2b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.083397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.084404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.085183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.085683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.100400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-93cf1b2b-f4da-4f0d-8e1c-2590523153a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:20.107096 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:20.108289 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.109111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:20.111826 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 177/368] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:20.113386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.113889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.114567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.136197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:20.137786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:20.169238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.217174 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 178/369] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:20.217648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07627-3edb-4196-ad56-733a67ea9f2b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.218007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.253636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.254013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.254841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.255342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.255888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.256749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:20.257157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.259207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a733407-c928-42ce-9635-92104e2ec624 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.273499 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 192/370] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:20.275553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e93012-4694-4b45-9bab-c00fab85b0b2 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.276089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e93012-4694-4b45-9bab-c00fab85b0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.276561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e93012-4694-4b45-9bab-c00fab85b0b2 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.278315 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 193/371] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:20.279248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e93012-4694-4b45-9bab-c00fab85b0b2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.339164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1c948-260d-49b7-ac8d-91bec7efdce2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.382273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.383516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.384023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.384909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.386018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:20.386519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.388354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93ba50-a313-4f3c-9f1d-5ae62a91dd6d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.403616 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 179/372] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:20.403616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.403616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.403616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.427775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:20.429860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:20.435667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e93012-4694-4b45-9bab-c00fab85b0b2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.469927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.480599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.485960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.485960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.485960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.494977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.496735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:20.515724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:20.516681 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:20.517529 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:20.517529 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:20.517529 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:20.517529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 194/373] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:20.518482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.518849 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 180/374] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:20.524894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.544099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.642029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d357-eea0-4996-91e5-251d794a6c48 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.655678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95a593-eee2-45e1-aba7-576fc4b69f20 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.689004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.690322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.691193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.692112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.697686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.698380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.698655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.698976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.707331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.709520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.711409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:20.718231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:20.719366 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:20.720409 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:20.726089 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 181/375] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:20.726089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:20.735666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:20.736601 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:20.737473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:20.737473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:20.737473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:20.739914 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 195/376] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:20.741247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.796026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.822103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:20.844645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccc9f5-f454-4f5c-a8ca-cfbdff354def None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.861115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed141ce9-c77e-402f-8f6a-bfcc99d21b95 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:20.885267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.885890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.886167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.886437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.887092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:20.887520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.888921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.898465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:20.899553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:20.899866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:20.900145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:20.900842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:20.901146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:20.902511 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 182/377] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:20.907807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:20.907909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:20.912497 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 196/378] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:20.913125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.014214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.020804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c491-a210-4725-9047-42fc412b53a0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.027913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89337df4-2578-4919-9fb3-0d6e93eb0722 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.057599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.058204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.058480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.058759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.069092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.069886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.070048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:21.070359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.070847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.071853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:21.072287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.073806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:21.075305 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.076367 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:21.077209 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 197/379] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.077873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.087776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.093712 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 183/380] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:21.100562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.191003 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd5a6-6a1b-4b1d-8071-231fbc2e621c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.210819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43f6e-3f0c-4c0b-bd6e-d116f771bb1a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.226732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.228310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.228690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.229055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.237279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.238983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.248911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.249548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.249831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.250109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:21.252156 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.253212 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 198/381] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.254308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.266042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.267558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.270989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:21.283049 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.293361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:21.294208 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:21.294208 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.294208 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:21.294208 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 184/382] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.294208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.338576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.355741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e30ac3-e448-451b-8914-c785b5c8afcf None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.384148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255764-f89c-4037-82ea-cbd149d52da7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.389743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.406390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae565f-8eff-442d-9284-a1619847198a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 199/383] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e839d1-1f8a-41a7-9877-f5540a995770 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e839d1-1f8a-41a7-9877-f5540a995770 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e839d1-1f8a-41a7-9877-f5540a995770 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 200/384] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:21.406867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e839d1-1f8a-41a7-9877-f5540a995770 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.421933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.422569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.422854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.423127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.431141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.432681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:21.445740 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.446762 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 185/385] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.448075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.490189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.514364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e839d1-1f8a-41a7-9877-f5540a995770 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.547773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bfc-1135-401a-adc3-810363a1906b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.549284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.549886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.550169 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.550440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.551175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:21.551480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.552995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.558923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.581733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.582354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.582641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.582917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.583565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:21.583910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.585302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da2609-4120-4336-8e72-574d408e7417 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.591926 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 201/386] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:21.592709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.602869 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 186/387] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:21.604233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.604524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.604832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.623433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:21.624673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:21.692483 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 187/388] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:21.693347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.697205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e30340-2a58-437b-934c-e26315a708fd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.744967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.745718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.746073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.746354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.748866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.757589 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.759101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:21.774449 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.775472 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:21.776412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:21.776412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.776412 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:21.776412 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 202/389] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.776634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.803000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:21.806295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c80941-7902-4eda-8857-3ef6002bdc02 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.847940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.848553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.848839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.849110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.850102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:21.851153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.853954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.892100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1bbf1-fc22-4d3d-81ac-31dbf570cacc None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:21.894919 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 188/390] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:21.895768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:21.935537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:21.936596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:21.937077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:21.937688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:21.949427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:21.951168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:21.966920 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:21.968027 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:21.969318 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:21.969318 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:21.969318 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:21.969318 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 203/391] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:21.969318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.003147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b517-7b33-48a1-9e05-00b97f7bd027 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.010981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.039457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.040109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.040390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.040666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.047943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.049389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:22.061783 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.062841 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 189/392] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.063784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.072959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5281-90d2-4e16-9bf8-cd11a86b42d5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.083158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.107431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.107988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.108271 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.108550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.109202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.109508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.110904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.123346 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 204/393] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:22.124189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.165720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd13fc-5375-4c08-9d9b-a392a23ac4c6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.200732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.201772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.202054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.202329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.213307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.214764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.222637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.226482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d508-ce65-479f-ba3b-c8cfc14cf253 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:22.229782 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.232705 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:22.233917 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:22.233917 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.233917 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:22.233917 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 190/394] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.233917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.263232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.263892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.264172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.264449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.276148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.278516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:22.295007 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.296008 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:22.296850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:22.296850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.296850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:22.296850 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 205/395] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.297599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.302482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.332627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dc771-b703-4821-b78d-8229aa51acd3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.413523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.414168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.414348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.414653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.415308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.415648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.417057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.428937 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 191/396] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:22.429763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.450678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca3d3-5488-46ed-81e9-372234324411 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.484621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.485137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.485416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.485720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.486450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.486754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.488226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.506806 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 206/397] 10.222.0.26 () {56 vars in 1139 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:22.507847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.521818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.530746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629d113-14d4-4d19-ae5e-c2921bcbad98 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.559751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.565543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.566186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.566547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.566835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.574632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.576139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:22.587629 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.588585 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:22.589477 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 192/398] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.590476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.611774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c270a3-f190-4008-9a4a-677b5e214570 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.647601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.648276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.648560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.648843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.649492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.649798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.651211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.662389 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 207/399] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:22.663204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.694407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec825-5e35-4403-9dbd-8b64479a701e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.727889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.728503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.728783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.729057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.729737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.729990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.730106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.731408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.743619 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 193/400] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:22.744625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.767205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372651-590f-4e82-afe0-78f3946493ec None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.798417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.806037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.806693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.806971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.807244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.814929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.816476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:22.829547 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.830645 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 208/401] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.836705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.853982 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dba178-f608-4c4e-89a9-2604262944c4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.891230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.891910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.892190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.892463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.903309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.904846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:22.921596 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:22.922856 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:22.923790 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:22.923790 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:22.923790 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:22.923790 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 194/402] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:22.923790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:22.947520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13150146-3e92-4481-8840-0e488bb72c0c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:22.975164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:22.984722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:22.985459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:22.985809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:22.986154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:22.986931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:22.987245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:22.988808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.009028 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 209/403] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:23.010670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.031373 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b9e5-2ea5-43eb-83e5-e4ba8c861407 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.033090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:23.069561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.070217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.070503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.070822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.078574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.080186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:23.094670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:23.095557 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 195/404] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:23.096062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.123370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ac550-d7fa-4939-87c4-60901127a115 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.160761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.161368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.161649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.161920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.162566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.164044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.205868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824fef2-d6c7-4dc2-af67-ef2b018bc8e8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.238600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.239262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.239547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.239856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.240557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:23.240893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.242344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.261433 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 196/405] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:23.262355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.287551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:23.292319 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 210/406] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:23 2022] POST /identity/v3/users => generated 247 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:23.293174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.359349 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:23.381296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a3de3-6a8f-46d3-a548-0a1b632c3570 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.424858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.425546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.425765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.426049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.426744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:23.427055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.428537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.451379 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 197/407] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:23.451379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.463984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9a6cb-7412-470c-823d-df23e0fe5e26 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.508535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.509203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.509770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.509770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.529686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.531165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:23.549910 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:23.550907 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:23.552349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:23.552349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:23.552349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:23.552349 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 211/408] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:23.552706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.577234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:23.597421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdeaf3-c608-4c17-a03c-6608d1db9f4f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.640431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.641050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.641323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.641921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.642287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:23.642629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.644143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.659557 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 198/409] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:23.660560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.677701 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:23.683152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5dfb-ae23-492c-a55d-ae9904a7671b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.730302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.730840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.731117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.731402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.752069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=eacd66576ed2425a8ad1fe0b69a4c5eb, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.754590 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.796006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b87a-808f-4906-afd4-19c9c9e08ab7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:23.835920 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 212/410] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:23 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:23.837486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:23.837669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.838459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:23.838837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:23.839203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:23.848050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:23.849747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:23.867704 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:23.868736 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:23.869767 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:23.872126 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 199/411] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:23.873939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:23.877937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.004861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bfa45-03b4-420b-ae0c-1544a5752ecb None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.016890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171818-a91f-40c5-a4bb-5530d51d370f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.058254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.059637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.060424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.061136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.062187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:24.062735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.064703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.064824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.065504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.065785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.066062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/aa1b9f055f6c4def891834a8114b04b8/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.081768 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 213/412] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:24.081962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=aa1b9f055f6c4def891834a8114b04b8, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.083051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:24.083385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.087994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.162060 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 200/413] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:23 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/aa1b9f055f6c4def891834a8114b04b8/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:24.172835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:24.253573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11af2c3-cbcc-416e-a721-287b01901ca0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.297570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.358933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f150777-b76c-4ac6-b38b-c635d1319dd5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.362070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.362711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.362996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.363275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.364013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:24.364326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.365795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.386898 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 214/414] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:24.392483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:24.408818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.410433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.411012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.411467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.426099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.431641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:24.451031 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:24.452023 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:24.452898 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:24.454836 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 201/415] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:24.460695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:24.473603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.543671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aab1ab-34bb-4ce3-82ed-1779f64864b1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.587210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.587923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.588267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.588553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/c6194924f3054c9598c3548d8e87b294/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.606263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=c6194924f3054c9598c3548d8e87b294, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.607753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.631258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59398-e228-408a-b385-6eb8243f1ddf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.646010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.669924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.669924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.669924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.670255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.673480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:24.673480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.673480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3bd65-3c2b-4c33-ac1a-c8f181529095 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.686473 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 215/416] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:24 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/c6194924f3054c9598c3548d8e87b294/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:24.699188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 202/417] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:24.705219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:24.819712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba8c9b-a08b-4d90-b3e0-5bdd0e41104d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:24.837865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:24.868101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:24.868814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:24.869100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:24.869380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/5de8280cded549c58eeabd4e4dc37964/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:24.896203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=5de8280cded549c58eeabd4e4dc37964, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:24.897709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969fcc-32d2-4c8d-83dd-464710f76037 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:24.972276 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 216/418] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:24 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/5de8280cded549c58eeabd4e4dc37964/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:25.840390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900347de-f293-4740-995b-bb7f89b2a466 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:25.840903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900347de-f293-4740-995b-bb7f89b2a466 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:25.840976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900347de-f293-4740-995b-bb7f89b2a466 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:25.842600 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 203/419] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:25.848322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aade6-e5ec-4b41-bb0d-6e09b567904d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:25.848801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aade6-e5ec-4b41-bb0d-6e09b567904d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:25.848875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aade6-e5ec-4b41-bb0d-6e09b567904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:25.869189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-409aade6-e5ec-4b41-bb0d-6e09b567904d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:25.870420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-409aade6-e5ec-4b41-bb0d-6e09b567904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:25.971414 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 217/420] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:26.430923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900347de-f293-4740-995b-bb7f89b2a466 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:26.441890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12974dd7-91d8-47ca-8fb5-7e01fbb11b5d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:26.442203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12974dd7-91d8-47ca-8fb5-7e01fbb11b5d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:26.442461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12974dd7-91d8-47ca-8fb5-7e01fbb11b5d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:26.449682 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 218/421] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:26.451015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:26.451305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:26.451597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:26.493177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:26.494396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:26.585584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900347de-f293-4740-995b-bb7f89b2a466 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:26.624487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:26.625137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:26.625419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:26.625698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:26.638075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:26.642951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:26.643899 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:26.644692 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 204/422] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:26.650688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:26.656919 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 219/423] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:26.684348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:26.757599 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd281-f392-406c-b9a2-5a001527c0b3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:26.795713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:26.796585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:26.798395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:26.798828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:26.799698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:26.800191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:26.801738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef8d5-ac95-4678-a524-5782e179ac54 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:26.811470 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 205/424] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:26.817390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:26.854741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:26.959358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1ac0-f7d9-45e8-80ee-72779c45eaae None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:26.998224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:26.999119 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:26.999549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.000053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.017887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.019641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.028235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff64c08-e35d-4cce-8b83-829d49ba33f4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.028235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff64c08-e35d-4cce-8b83-829d49ba33f4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.028838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff64c08-e35d-4cce-8b83-829d49ba33f4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.030227 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 206/425] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:27.041084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.041084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.041084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:27.044555 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:27.045666 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:27.046731 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:27.046731 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:27.046731 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:27.049493 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 220/426] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:27.056145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.074086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:27.079415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:27.169501 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 207/427] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:49:27.170589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.183973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.209570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e66560-38c0-47bd-985c-0fbee9b794d1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.226336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.253955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.254867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.255298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.255755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.256566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:27.257020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.259258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.281643 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 221/428] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:27.287525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.368896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbd289-9802-4462-a18b-f49cf58fc108 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.408472 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.409123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.409516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.409844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.428617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:27.438028 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:27.439012 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:27.439857 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:27.442073 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 208/429] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:27.447908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.490736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edf02-0770-4355-90ca-e9fa5ff87cb9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.531370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.532257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.532676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.532954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.536110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.541588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.543046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-4adc0394-753b-49c9-a5e1-259b263dc6ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:27.557587 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:27.558579 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:27.559569 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 222/430] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:27.561980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5bb0d-d021-49ce-bdaf-d0bca484efe7 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.561980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5bb0d-d021-49ce-bdaf-d0bca484efe7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.561980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5bb0d-d021-49ce-bdaf-d0bca484efe7 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.564828 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 223/431] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:27.566225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5bb0d-d021-49ce-bdaf-d0bca484efe7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.570815 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.589335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1856d-40d4-4b62-8a6a-dedda5619fa9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.628275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.629023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.629308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.629584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.630258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:27.630570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.631986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.641219 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 209/432] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:27.642020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.693131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5bb0d-d021-49ce-bdaf-d0bca484efe7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.729229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.729869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.730160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.730438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.741533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-f95874c0-a16c-4536-a5f4-2c6f94c3ef62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:27.746982 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:27.748197 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:27.749002 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 224/433] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:27.752962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061d27-2cf9-408c-985e-f24c91c99b3f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.753495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061d27-2cf9-408c-985e-f24c91c99b3f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.753773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061d27-2cf9-408c-985e-f24c91c99b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.756899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94472389-99a2-43c6-8e4d-123386377ad6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.773204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c3061d27-2cf9-408c-985e-f24c91c99b3f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:27.774483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c3061d27-2cf9-408c-985e-f24c91c99b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:27.792508 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.793130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.793413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.793694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.794429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:27.794737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.796207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.814265 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 210/434] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:27.815114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.820021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.828064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:27.845029 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 225/435] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:27.846293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b44db6-f66f-44d4-9431-3d374d49f3a6 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.846584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b44db6-f66f-44d4-9431-3d374d49f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.846862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b44db6-f66f-44d4-9431-3d374d49f3a6 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.848306 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 226/436] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:27.848848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b44db6-f66f-44d4-9431-3d374d49f3a6 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:27.926338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17508ff-c3d5-4301-88b8-017773c60046 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.955785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b44db6-f66f-44d4-9431-3d374d49f3a6 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:27.966635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.967286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.967930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.968214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.979452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.980983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.993616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:27.994249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:27.994603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:27.994944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:27.995813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:27.996194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:27.998087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fa95d-36d7-4f43-8b7a-beb3378b2a39 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:27.998392 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:27.999448 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:28.006173 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:28.006173 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:28.006173 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:28.006173 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 211/437] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.006173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.020568 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 227/438] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:28.020568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.020568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.020568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.037880 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:28.039918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:28.083312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.090840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.115380 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 228/439] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:28.121932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.134890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74537af0-4de2-4675-8523-5641a21d0132 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.170645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.171451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.171866 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.172236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.184494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.186239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:28.202009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.203117 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:28.204150 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:28.204150 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:28.204150 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:28.204150 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 212/440] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.204736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.247471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b6880-d2ef-458a-a033-95fe21ccb673 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.283434 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.284373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.284832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.285251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.286048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:28.286501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.288105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.308977 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 229/441] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:28.310120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.323231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc02fe-50f5-4d46-8f43-2b58dc7242a4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.333795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.354113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.362097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.362733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.363016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.363294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.375537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.377035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:28.392263 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:28.394130 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:28.394130 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:28.394130 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:28.394130 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 213/442] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.395227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.421937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4305-b429-4f7f-a014-ee10fc478cb6 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.456233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.456884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.457167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.457445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.468462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:28.473707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.475538 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 230/443] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.477693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.502741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670114-053d-4b10-b921-3ea8f72f2e86 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.536726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.537337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.537619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.537900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.538551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:28.539420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.540845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.552983 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 214/444] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:28.553429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.575486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.577420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.581437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bfef-4a47-46c6-a634-a665d391e44e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.616098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.616727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.617038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.617318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.625088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.626548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:28.639401 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.643033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 231/445] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.644180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.656197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44e14-3a31-4125-9380-3e787f9bf4e9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.690884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.691522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.691838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.692117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.702236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:28.707017 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.707949 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:28.708799 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 215/446] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.709460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.755122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2eca1-dd42-44a6-9a8f-e3caad87f4b7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.790606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.791241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.791522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.791868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.792555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:28.792857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.794002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.794278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.806356 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 232/447] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:28.807190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.814996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d4e5-f4ce-425d-aff9-7477eb5883d5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.827494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:28.850489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.851101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.851389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.851696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.852366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:28.852674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.854144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.862214 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 216/448] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:28.864252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:28.910519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef491a3-6679-4773-9d38-b6d363f2fd02 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.945548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:28.946277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:28.946558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:28.946821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:28.953464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:28.954903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:28.967065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92dd0f-ac52-46de-9849-fede2a808f0e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:28.967258 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:28.968562 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 233/449] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:28.969420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.003284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.003967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.004303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.004587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.005401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.005656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.007085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.021609 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.025915 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 217/450] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:29.026727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.055362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.077272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be9b-d8d9-4ca4-9af1-7b115b3398aa None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.111923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.112539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.112819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.113095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.113766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.114072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.115444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.124087 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 234/451] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:29.124928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.127382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50f06-8a37-4258-a80e-685346e26be0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.161483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.162108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.162389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.162662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.169711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.171167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:29.183199 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:29.184259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 218/452] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:29.185159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.227477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b771-67b4-4280-9150-7e984204d9db None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.246928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.265447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.265703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.265987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.266262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.276427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.278548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.280069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.290514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec769e-7c6a-4abe-91c2-a37c582b6821 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:29.294593 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:29.295984 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:29.297022 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:29.297022 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:29.297022 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:29.297022 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 235/453] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:29.297022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.330852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.331475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.331779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.332058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.332765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.333068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.334596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.347456 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 219/454] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:29.348276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.404373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42a7e6-c1a6-4199-91ed-b561051e490f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.442405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.443028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.443314 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.443627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.450020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3fb4-be73-41aa-8a8d-15ded5f8b7a7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.471911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.485766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.486401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.486682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.488761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.497953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.499507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.502879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.513501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.514989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:29.515164 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:29.516252 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:29.517145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:29.517145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:29.517145 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:29.517145 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 220/455] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:29.517794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.628924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb5086-6de0-48a8-a9a3-a4b648c75b1f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.651825 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 236/456] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:29.652691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.672451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.673074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.673442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.673786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.674574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.674887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.676410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.694807 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 221/457] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:29.695716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.727057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.749432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.757992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea157a2-0a1a-4a23-8326-358c77ee5c53 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.793054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.793582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.793862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.794142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.794790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.795102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.796516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.799803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d044f-bb3b-472b-b96b-64b94e08cbb0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.808584 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 237/458] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:29.809451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.835618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.836325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.836602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.836875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/d6a6265c254040d9a0d8ae45b29d3073/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.853053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=d6a6265c254040d9a0d8ae45b29d3073, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.854541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:29.940350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7271417-35f6-420d-90dc-f71f76cd2ed4 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:29.942462 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 222/459] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:29 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/d6a6265c254040d9a0d8ae45b29d3073/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:29.943330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:29.990741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:29.991552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:29.991876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:29.992159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:29.992836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:29.993146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:29.994744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.019843 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 238/460] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:30.019843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.036478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.057806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.081933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca21ae-3f3c-40ce-b6b6-6c696c45485c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.125084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.125855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.126140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.126421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.139800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.141279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.149397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8269b-a828-486d-bf42-98554ff32a87 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:30.166677 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:30.167866 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:30.168732 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:30.168732 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:30.168732 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:30.168732 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 223/461] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:30.169016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.195139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.195794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.196077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.196350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.287540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae5d9-9626-44cc-9980-9cf89c2dd784 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.307524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.310071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.327058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.332650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.333314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.333585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.333862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.339288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.343371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.344879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:30.359228 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:30.360514 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:49:30.361615 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:30.370439 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 224/462] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:30.370439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.450184 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 239/463] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:30.451206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.487698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff86a-8128-4357-a50c-e505c5221663 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.528782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.529429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.529722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.530001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.539511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.541008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:30.552631 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:30.553547 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:30.555075 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 225/464] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:30.556036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.564447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b213b-ebf0-4da8-b8a8-508a2d3b90cd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.577848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.598975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.599614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.599902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.600181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.602251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:30.602251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.602982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.614587 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 240/465] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:30.615435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.618158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.675213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e1bb2-8e8b-4cf8-b36e-eb574440a734 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.711231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.711885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.712171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.712439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.713182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:30.713494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.714941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.735263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1485867-7719-4de6-a33b-d002a98d9e13 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.735481 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 226/466] 10.222.0.26 () {56 vars in 1139 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:30.736146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.774867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.775506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.775815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.776107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.787019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.788496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:30.803682 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:30.804659 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:30.805537 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:30.805537 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:30.805537 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:30.805537 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 241/467] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:30.805758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.839836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.854104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7696-2568-483f-ab75-5011ea75bd4a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.866565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:30.889891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.890499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.890776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.891047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.891796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:30.892098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.893516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.919305 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 227/468] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:30.919305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:30.927166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395b73-5b98-4c35-b9d2-ac623a61141f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:30.963753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:30.964395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:30.964683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:30.964963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:30.971814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:30.973277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:30.985257 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:30.986293 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:30.987176 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 242/469] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:30.987683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.025140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86bb5d-f24b-480d-ae01-0572fbcfd4e9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.062680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.063287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.063563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.063875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.078747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.097818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647da9-b944-44a9-8fee-6d7d4e601848 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.104537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.133909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.134519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.134797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.135071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.135789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:31.136093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.137501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.142064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.143513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.154429 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 243/470] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:31.154429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.258302 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 228/471] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:31.259382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.260187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea3d25-0fc5-4b5e-8bd5-f10f75f2ac9f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.300838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.301556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.301906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.302243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.318250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.333289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.367759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b928d8-43ed-4edd-8216-5a816789bbf1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.376156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.378200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.406007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.406928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.407214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.407673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.408363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:31.408668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.410100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.426428 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 229/472] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:31.426428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.489474 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 244/473] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:31.490258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.536344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f2ce2-b1a7-4487-97b5-eb6bbb36055a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.539301 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.565181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.573990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.574628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.574911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.575187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.582314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.583802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:31.595300 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:31.596385 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:31.597244 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:31.597657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2c44-1022-4591-9b32-16b9a7a56222 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.601301 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 230/474] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:31.602209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.636158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.636854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.637135 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.637404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.648086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.649602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:31.664972 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:31.665932 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:31.666773 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:31.666773 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:31.666773 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:31.666773 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 245/475] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:31.667423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.727262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324d87d-7397-4487-9097-293bbbdd3ac5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.766038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.766654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.766931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.767213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.767984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=eacd66576ed2425a8ad1fe0b69a4c5eb) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:31.768292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.769823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.770261 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.776139 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 231/476] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/role_assignments?user.id=eacd66576ed2425a8ad1fe0b69a4c5eb&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:31.776782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac54fa-8c27-4fb0-9157-7b6a2cc6b301 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.776958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.829248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.830164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.831367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.831870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.848789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.850627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.853350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:31.877013 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:31.878325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:31.879371 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:31.879371 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:31.879371 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:31.883499 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 246/477] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:31.884521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:31.930065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad633-fa2c-4c59-8a9d-262ad2f573df None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:31.978077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:31.978694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:31.978976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:31.979248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:31.980394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:31.980394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:31.981697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.001405 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 232/478] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.002762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.013583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2d7fa-7ebb-486d-98d7-111da9be2ddc None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.059972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.060966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.061470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.061939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.062793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.063605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.065907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.079075 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 247/479] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.085825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.097491 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.133704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb60ee1-347e-44bb-baf6-936cfde94b9e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.150355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.171250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.171894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.172162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.172441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.173089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.173389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.174768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.188642 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 233/480] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.189583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.196586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3951e-90f8-4ea0-a89e-7a4907863ab7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.232420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.233433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.233778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.234145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.234944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=aa1b9f055f6c4def891834a8114b04b8) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.235321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.236891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27048af-4c97-4fd0-970f-71e286248fbc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.243619 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 248/481] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/role_assignments?user.id=aa1b9f055f6c4def891834a8114b04b8&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.244326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d37fb-296e-428d-8a5f-2be94290de59 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.244611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d37fb-296e-428d-8a5f-2be94290de59 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.244884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d37fb-296e-428d-8a5f-2be94290de59 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.246287 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 249/482] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:32.251307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d37fb-296e-428d-8a5f-2be94290de59 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.330186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a55bd9-df24-4cee-80b9-4664374defd2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.352419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.373051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.374395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.374796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.375133 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.384189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.385904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.389823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d37fb-296e-428d-8a5f-2be94290de59 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6964ad00-d34b-4dbd-b932-4d88c8fe34ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:32.401209 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:32.402325 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:32.403333 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:32.407870 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 234/483] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:32.407870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.407870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.407870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.427884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:32.429284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:32.441458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.442395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.442681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.442954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.452112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.454870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.466653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:32.469378 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:32.470234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:32.471066 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 250/484] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:32.471815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.532197 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 235/485] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:32.533291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.591079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee0f3-def4-4fc4-9385-df915f89fb85 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.622145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.630234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.631246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.631772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.632763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.633699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.634166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.635807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.650982 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 251/486] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.660237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa2931-6e77-4b78-911f-01c7e6e97c76 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.663828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.719201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.719782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.720066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.720340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.721615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.721615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.722901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.752267 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 236/487] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:32.759140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.786696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.807311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff2a9-01d0-418d-b821-b897cf13df3d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.879251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.879922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.880214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.880483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.881231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=c6194924f3054c9598c3548d8e87b294) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.881540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.883696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.884750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a98174-8ad8-44b2-98ad-3b13bb06a4b9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:32.891448 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 252/488] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/role_assignments?user.id=c6194924f3054c9598c3548d8e87b294&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:32.892328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:32.954220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:32.958540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:32.959168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:32.959448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:32.959748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:32.960525 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=5de8280cded549c58eeabd4e4dc37964) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:32.960838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:32.962415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:32.975225 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 237/489] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/role_assignments?user.id=5de8280cded549c58eeabd4e4dc37964&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:33.002419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424874-6df0-46ee-ade2-62006e05934f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.044546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.045232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.045511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.045782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.065990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:33.072325 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:33.073276 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:33.074067 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 253/490] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:33.079837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.224015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:33.236736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a122d-50c1-4062-8005-e40ad71ce6e5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.273768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.274381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.274667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.274941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.275634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:33.275943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.277325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.286713 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 238/491] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:33.293690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.440794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673ef2f-f130-4adb-9078-058d18f3df04 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.462516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:33.490014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.490591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.490874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.491150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.509795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.511278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:33.532452 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:33.533574 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:33.534494 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:33.534494 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:33.534494 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:33.534661 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 254/492] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:33.539921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.682226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f119-4516-425e-9a29-c49dbe093d32 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.728809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.729433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.729717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.729996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.730649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:33.730957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.732347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.744966 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 239/493] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:33.750684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.922082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0458c-ca88-4933-9ea4-5329dca9559d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.962425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.963070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.963353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.963654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.971836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.973322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:33.985298 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:33.986352 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:33.987388 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 255/494] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:33.993079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.024059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.130568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911d2db-3a7d-4838-a2d3-ae8c305cb8ef None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.183979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.184618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.184903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.185184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.185949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:34.186259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.187750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc34c0-6365-4c9f-9c8f-a7f18bb3b562 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.206961 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 240/495] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:34.212647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.269947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.360779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13139f21-270e-4035-8ec4-ca598e5fbb18 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.369868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5fa6a-4c79-439b-844f-3d2665c57a54 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.370397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5fa6a-4c79-439b-844f-3d2665c57a54 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.370397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5fa6a-4c79-439b-844f-3d2665c57a54 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.373294 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 241/496] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:34.379035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.379744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.380039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.414967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:34.416189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:34.430860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.431483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.431787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.432049 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.540634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.556023 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 242/497] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:49:34.564854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.566354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.607394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.686772 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 256/498] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:34.692029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.848302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31317deb-e491-4a95-8791-9f90edf81043 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.901009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.901651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.901926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.902196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.916401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.917861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:34.935646 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:34.936565 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:34.937789 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:34.937789 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:34.937789 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:34.937789 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 243/499] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:34.942252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.020000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.099452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f9769-ef06-404e-a522-9e2efc6a91e6 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.112502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.127733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.136198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.136896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.140263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.140263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.140263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:35.140263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.140263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347adc8-e9e5-4013-aec9-154cda2b230a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.155773 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 257/500] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:35.157338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880fba-7d61-478d-8ea4-eee79e4bc529 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.157767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880fba-7d61-478d-8ea4-eee79e4bc529 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.158168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880fba-7d61-478d-8ea4-eee79e4bc529 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.162641 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 258/501] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:35.164842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880fba-7d61-478d-8ea4-eee79e4bc529 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.178304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a60c2-538a-4520-b2ec-99f4fe2d8f25 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.221595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.223187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.223698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.224313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.238694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-122223f2-7784-473e-8235-d80e4678dbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:35.246033 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:35.247460 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:35.248977 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:35.251635 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 244/502] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:35.253232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.253697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.254139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.276370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:35.278658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:35.301990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880fba-7d61-478d-8ea4-eee79e4bc529 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.348643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.349866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.350386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.350846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.360019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.361209 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 245/503] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:35.362278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.367857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:35.390385 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:35.391326 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:35.392435 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:35.396400 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 259/504] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:35.403082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.442998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.501362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.567294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326116a-2494-4bf4-9c06-5e15e7b8351a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.610801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29294e2-19c5-4b70-86d9-908285ab42b1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.649140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.650974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.651861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.652794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.654113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:35.655116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.657093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.659999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.660957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.661445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.661912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.662820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:35.665155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.666839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9c653-34a3-43f0-9db0-33d30138ba01 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.682401 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 246/505] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:35.690075 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 260/506] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:35.692746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.705007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d1df-a759-4597-bbc5-68faf607e34b None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.705629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d1df-a759-4597-bbc5-68faf607e34b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.706268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d1df-a759-4597-bbc5-68faf607e34b None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.707761 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 261/507] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:35.708381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d1df-a759-4597-bbc5-68faf607e34b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.801520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.805015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ac4b0-d391-49c0-a8b7-a2172cc5f576 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.819499 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d1df-a759-4597-bbc5-68faf607e34b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.829591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:35.840772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.841413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.841693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.841971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.852423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.853855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.854905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.856853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.856853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.856853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.856853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=d6a6265c254040d9a0d8ae45b29d3073) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:35.857146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.858521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e9811-aa0c-476b-8a74-96851dd96481 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.864229 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 262/508] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/role_assignments?user.id=d6a6265c254040d9a0d8ae45b29d3073&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:35.865364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc4e7-8d36-42db-9283-f0457c57dc3f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.865643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc4e7-8d36-42db-9283-f0457c57dc3f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.865930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc4e7-8d36-42db-9283-f0457c57dc3f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.867883 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 263/509] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:35.868963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa06d-ddee-4cfe-a999-d2412c076b3e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.869237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa06d-ddee-4cfe-a999-d2412c076b3e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.869505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa06d-ddee-4cfe-a999-d2412c076b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:35.873468 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:35.874435 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:35.875362 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:35.875362 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:35.875362 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:35.875362 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 247/510] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:35.876542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.896799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-afafa06d-ddee-4cfe-a999-d2412c076b3e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:35.899739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-afafa06d-ddee-4cfe-a999-d2412c076b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:35.979367 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 264/511] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:35.980513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.980794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.981070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.002626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d46cd-4104-4076-a88d-948dd3ae67c5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.004669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:36.005932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:36.053060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.053638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.053962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.054239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.067359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.070587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:36.088445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:36.089579 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:36.090465 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:36.090465 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:36.090465 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:36.090465 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 248/512] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:36.091046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.095194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.111622 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 265/513] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:36.121077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.159324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.233891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca684112-4d61-480b-ab4a-0ed28e2b5f0b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.257497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068c9c5-4b33-49e5-bd71-b9faba6c5cea None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.280051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.280868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.281150 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.281875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.282827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:36.283147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.284810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.302467 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 249/514] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:36.303641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.304706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.305420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.305778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.306128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.306892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:36.307274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.309193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.322656 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 266/515] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:36.323688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.389312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.437686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b694d-de3f-438e-8800-b03e0b96f497 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.459549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.468366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12ceb-3d85-4024-941a-6ff274afe116 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.474831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.476191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.476630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.476923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] PATH_INFO: `/v3/services/network` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.477631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.479008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:36.490320 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:36.492067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:36.492951 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 250/516] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:36.494413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.511287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.511956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.512245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.512525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.520518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.521989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:36.533418 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:36.534402 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:36.535460 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:36.539862 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 267/517] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:36.540545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.609746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441cfb1-3b81-4206-b415-7e3df8114bda None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.637960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.646371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.647057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.647345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.648147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.648797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.650703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bbd21-3af1-4505-9290-3e7c1fcf1049 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.650809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.688246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.688917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.689203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.689480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.690072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.691458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:36.703470 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:36.704523 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:36.705373 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 268/518] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:36.706215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.725119 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:36.777872 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 251/519] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:49:36 2022] POST /identity/v3/users => generated 254 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:36.779383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.829437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6afe-83d3-494e-bdf1-1acd01a2d433 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.872634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.873236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.873521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.873847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.874626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] RBAC: Adding query filter params (name=network) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:36.874941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.876752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.887303 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 269/520] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:36.887303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:36.933199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098e4f0-3263-408a-8df7-765c6b24c2da None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:36.976808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:36.977446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:36.977956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:36.978113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:36.978871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:36.979334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:36.980982 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:36.994601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:37.002532 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 252/521] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:37.003602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.019376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c9148-ddd7-4f2b-b17e-589ae5f31044 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.057316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:37.057537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.057942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.058227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.058511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.059397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:37.059509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.060834 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.067652 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 270/522] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:37.067652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.151307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ef75-5520-4a1b-867a-b05e3d303bbf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.181270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0c56-3414-4cd8-a038-4430b736d1bd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.197797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.198787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.199138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.199483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.200535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] RBAC: Adding query filter params (type=network) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:37.200916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.202813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.210911 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 253/523] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:37.212875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.223196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.224182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.224611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.225039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.237825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.239567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.247627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:37.273311 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:37.274496 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:37.275600 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:37.275600 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:37.275600 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:37.284082 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 271/524] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:37.302393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.353880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1037-a723-4860-aae4-8c43a9932d31 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.363425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:37.399616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.400635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.401130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.401670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.402622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:37.403558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.407048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.418537 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 254/525] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:37.530778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5e4f-91af-42cd-b85a-7597aa490b83 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.582486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.583125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.583408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.583721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.584385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:37.584690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.586650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.602841 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 272/526] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:37.610486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.725589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc3c5b-c585-42e9-8223-6eef07b9ab22 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:37.765648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:37.766655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:37.767186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:37.767702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:37.775303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:37.777412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:37.790155 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:37.791188 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:37.792126 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:37.797626 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 255/527] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:37.800992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:37.906484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:37.965736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcbcf-dfd3-4e56-b562-5a269d4db4fe None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:38.016534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:38.017429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:38.017866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:38.018287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:38.019144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:38.019629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:38.021225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ff03-8b6c-4dfb-b36c-cb1663c5748f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:38.038980 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 273/528] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:38.047620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:38.089374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:38.191076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d6b1-42e7-4188-b92c-ba21d8cf57ee None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:38.245475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:38.246089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:38.246624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:38.246901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:38.248886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=eacd66576ed2425a8ad1fe0b69a4c5eb, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:38.250463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc696b-946a-479f-81b0-a77b20c133ae None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:38.336728 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 256/529] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:38 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:38.573725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4ff5-f353-49c3-a77c-5fc0dfe46bec None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:38.574254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4ff5-f353-49c3-a77c-5fc0dfe46bec None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:38.574254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4ff5-f353-49c3-a77c-5fc0dfe46bec None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:38.575864 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 274/530] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:38.581602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bee7-29e6-4cec-a0cf-f5be9583e575 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:38.581896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bee7-29e6-4cec-a0cf-f5be9583e575 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:38.582202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bee7-29e6-4cec-a0cf-f5be9583e575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:38.604546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7396bee7-29e6-4cec-a0cf-f5be9583e575 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:38.609346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7396bee7-29e6-4cec-a0cf-f5be9583e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:38.713638 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 257/531] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:39.045797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d4ff5-f353-49c3-a77c-5fc0dfe46bec None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:39.187285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d4ff5-f353-49c3-a77c-5fc0dfe46bec None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:39.208995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:39.229308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:39.229954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.230235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.230509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.242088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:39.247323 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:39.248288 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:39.249129 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 275/532] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:39.253168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b91cb-668f-4abf-be13-e93bda8b29d9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.253462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b91cb-668f-4abf-be13-e93bda8b29d9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.253742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b91cb-668f-4abf-be13-e93bda8b29d9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.255203 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 258/533] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:39.258779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:39.261753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.262311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.262655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.296797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:39.298040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:39.325989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:39.389249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd03d9a-5e4c-4d9a-a4b3-2b6131a3d497 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:39.418901 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 259/534] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:39.432216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:39.432842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.433124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.433398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.434077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:39.434384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:39.435789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:39.445111 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 276/535] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:39.450726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:39.567774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1eb56-9bc4-44ce-84bd-a8d2bb8a8a00 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:39.605906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:39.606911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.607399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.607944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.619257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:39.621039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:39.637574 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:39.638583 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:39.639508 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:39.639508 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:39.639508 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:39.641752 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 260/536] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:39.648215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:39.708830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:39.790250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf7aac-63ef-4992-906c-611254c783b1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:39.842694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:39.847574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:39.865280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:39.866427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:39.866738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:39.867038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:39.867866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:39.868049 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:39.869344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:39.900238 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 277/537] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:39.915470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.018413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efb80-f132-4bae-9094-a2f9678dc20a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.058141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.059163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.059708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.060228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.073614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6c7148bc-eaf0-42d6-b744-b4941e1f1cbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:40.079625 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:40.080676 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:40.081513 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:40.084509 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 261/538] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:40.086828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3b39d-ce7e-45d2-886c-2528739d6556 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.087389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3b39d-ce7e-45d2-886c-2528739d6556 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.087905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3b39d-ce7e-45d2-886c-2528739d6556 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.089905 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 262/539] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:40.092011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3b39d-ce7e-45d2-886c-2528739d6556 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.095668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.100358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27ba4-f94f-40f3-bd0d-e649f49064ce None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.156542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.157553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.158039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.158515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.167571 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.169604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.177338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-d02018e8-fdc9-4b52-be94-261a94d80bdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:40.183194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:40.184206 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:40.185141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:40.189428 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 278/540] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:40.191653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72083a-ddbd-42cf-bb1b-f22022b023e8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.191653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72083a-ddbd-42cf-bb1b-f22022b023e8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.191653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72083a-ddbd-42cf-bb1b-f22022b023e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.212068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1f72083a-ddbd-42cf-bb1b-f22022b023e8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:40.213280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1f72083a-ddbd-42cf-bb1b-f22022b023e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:40.229542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3b39d-ce7e-45d2-886c-2528739d6556 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.267325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.267916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.268201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.268478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.269659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:40.269764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.271509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.283734 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 263/541] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:40.283734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.289899 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 279/542] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:40.291496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880bf2-f107-4b46-8488-32d50d884f83 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.292063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880bf2-f107-4b46-8488-32d50d884f83 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.292549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880bf2-f107-4b46-8488-32d50d884f83 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.294088 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 280/543] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:40.295064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5880bf2-f107-4b46-8488-32d50d884f83 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.401578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.415422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5880bf2-f107-4b46-8488-32d50d884f83 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.437945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac46d7-4a43-4d33-8576-38e125e34096 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.459219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.460044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.460329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.460602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.467841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.473118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.474904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.479916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.480380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.480768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.481141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.481986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:40.482413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.483957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-05f228cc-4d2a-4a64-89c1-caf2e6921cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:40.493769 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:40.494901 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:40.495845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:40.495845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:40.495845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:40.503182 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 281/544] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:40.504769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.505228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.505670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.506359 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 264/545] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:40.507896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.526091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:40.527784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:40.601321 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 282/546] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:49:40.602172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.620957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf710-18a0-44d3-8b56-b73e7e00d18c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.659764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.660752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.661222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.661669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.662512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:40.663125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.673109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.686910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.693581 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 265/547] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:40.695275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.729302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4de1-1fd3-42cf-bec5-5b89467c92f8 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.745451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.778549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.779295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.779670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.780018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.794575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.796255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.811266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9a27d-1561-40c4-be61-0d37db5f57fc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:40.817744 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:40.818693 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:40.819713 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:40.819713 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:40.819713 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:40.823735 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 283/548] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:40.824958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.865279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.867466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.867959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.868414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.869402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.871392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:40.940634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879b24-e450-4a13-9e68-344570cc0baa None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:40.963058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:40.974678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:40.975884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:40.976164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:40.976441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:40.981512 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 266/549] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:49:40 2022] POST /identity/v3/services => generated 240 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:40.981512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:40.986734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:40.988266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:41.002551 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:41.009248 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 284/550] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:41.010234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.118721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:41.133152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6d2d-cede-4195-a33c-a984d2dc4343 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.138766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded5535-147e-4c32-a918-722e6baa69b7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.178253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.178928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.179322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.179646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.180504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:41.180818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.182606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.183367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.184093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.184384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.186638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.186638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.186992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.196772 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 267/551] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:41.197392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.269645 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 285/552] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:49:41 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:41.270582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.321097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec121-5841-4c52-81f3-e6accc1ac03d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.339891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:41.379797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.380449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.380725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.381000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.381781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:41.382091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.383650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37757-cab2-400c-9877-4f492bd420b2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.429529 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 268/553] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:41.429529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1ff2-09a5-402e-b152-35a032aa0611 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.429529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1ff2-09a5-402e-b152-35a032aa0611 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.429529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1ff2-09a5-402e-b152-35a032aa0611 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.429895 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 269/554] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:41.432574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e1ff2-09a5-402e-b152-35a032aa0611 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.452796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:41.458247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b7955-083e-4fc7-bd1b-7fefb3c3cde7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.509113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.509907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.510197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.510463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.519545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.521101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-c23450d7-9fa4-42be-be50-49c2782940d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:41.535334 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:41.536360 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:41.537253 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 286/555] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:41.538660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.539703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.539703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.563908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:41.565116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:41.587839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e1ff2-09a5-402e-b152-35a032aa0611 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.635281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.636093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.636375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.636650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.648331 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 287/556] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:41.649599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.650039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.651511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:41.670725 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:41.671929 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:41.672887 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:41.672887 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:41.672887 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:41.680049 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 270/557] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:41.681293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.691762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:41.749950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:41.771996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0f907-5550-40f1-b113-78b1657d5ecc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.828714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabb6cc-baef-4f39-82a3-eb1703d0417e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:41.843267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.845859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.846383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.846866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.847993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:41.848677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.851966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.878689 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 288/558] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:41.886751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.901104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c692b76-4136-4512-b309-03c325759220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:41.901104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c692b76-4136-4512-b309-03c325759220 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:41.901711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c692b76-4136-4512-b309-03c325759220 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:41.901711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c692b76-4136-4512-b309-03c325759220 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:41.902283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c692b76-4136-4512-b309-03c325759220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:41.902594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c692b76-4136-4512-b309-03c325759220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:41.906124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c692b76-4136-4512-b309-03c325759220 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:41.930197 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 271/559] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:41.934679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c692b76-4136-4512-b309-03c325759220 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:41.998204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:42.003529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899204-2b86-4616-b32e-e9f423e9c6e5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.043386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.044847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.045484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.046094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/d6a6265c254040d9a0d8ae45b29d3073/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.048770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=d6a6265c254040d9a0d8ae45b29d3073, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.050658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.090342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c692b76-4136-4512-b309-03c325759220 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.115442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:42.127028 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 289/560] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:41 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/d6a6265c254040d9a0d8ae45b29d3073/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:42.133721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.133721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.133721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.133721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.135505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:42.142961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.144480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:42.162143 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:42.163905 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:42.165078 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 272/561] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:42.170560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:42.335802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac486c9-ac1a-467e-b8ff-f9ab5d8fc70e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.381075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ff66-467c-4b37-aab7-e4ae0b2dfc5e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.398390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.399392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.399919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.400389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.401284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:42.402044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.404135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.420267 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 273/562] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:42.425626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.426278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.426561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.426838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.429728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:42.442419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:42.448141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:42.449107 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:42.449953 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 290/563] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:42.455672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:42.472284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:42.510010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:42.584825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412920da-5bb1-494a-bf69-857ca709f951 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.614245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41fd3-eaaa-4877-8ce2-e87bee66a5d8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.657840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.658541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.658822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.659096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/27db563707034f9aa2e8df84704ff249/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.694393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=27db563707034f9aa2e8df84704ff249, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.696027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.698213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.699133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.699699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.700177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.703622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:42.704270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.705892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.719715 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 291/564] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:42.725927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:42.793496 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 274/565] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:42 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/27db563707034f9aa2e8df84704ff249/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:42.830625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:42.873814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3226e3-f625-4a57-b168-d67c500c9f83 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:42.929502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:42.930254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:42.930533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:42.930805 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:42.943436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:42.944990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:42.962018 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:42.963033 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:42.963974 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:42.963974 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:42.963974 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:42.963974 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 292/566] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:42.970395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:43.094664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:43.102548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd7b8-8bd4-4084-a832-3e8bb27b6fd5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:43.143218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:43.144275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.144763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.145236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.146097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:43.146626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:43.148934 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730339c-ebfb-435d-bac8-fdf283028953 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:43.168870 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 275/567] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:43.174700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:43.328385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047bbaf-dd08-41c5-9ac0-f971c81bcec2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:43.370185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:43.389798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:43.390826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.391339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.391849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.404813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:43.406616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:43.409476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154acb6-9919-4770-b39a-557497f225cc None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.409782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154acb6-9919-4770-b39a-557497f225cc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.410044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154acb6-9919-4770-b39a-557497f225cc None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.412875 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 276/568] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:43.417293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.417593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.417875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:43.421133 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:43.422090 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:49:43.423147 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:43.425665 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 293/569] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:49:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:43.431990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:43.464657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:43.465918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:43.595944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dc2a4-3c90-4652-9ca5-12993226f139 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:43.597429 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 277/570] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:43.651902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:43.652526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.652813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.653094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.653847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:43.654185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:43.655761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea19036-27ed-49f0-99e4-fce2b36f95c8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:43.691698 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 294/571] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:49:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:43.698350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:43.817309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:43.840233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece77ae8-b719-4aa8-86c8-ccbfd8842b9d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:43.843790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f5a1-a152-431c-8162-a8401b3f789d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.844262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f5a1-a152-431c-8162-a8401b3f789d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.844340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f5a1-a152-431c-8162-a8401b3f789d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.845934 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 295/572] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:43.851496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.851816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.852095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.878959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:43.879635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:43.879922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:43.880197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:43.892603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:43.893873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:43.950940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:43.952375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:43.963980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:43.991922 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 296/573] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:49:43.993038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.045433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.064615 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 278/574] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:44.070899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.179267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e02e7-1766-4b09-aa32-24c4dec8d992 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.220321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.221783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.222066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.222462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.223100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.224729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:44.236345 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:44.237508 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:44.238525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 297/575] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:43 2022] GET /identity/v3/services/network => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:44.244546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.283696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8867d-5737-4735-a333-717bd9e2dac9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.313402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.336504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.342615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.342615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.342615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.349779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.351876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:44.367962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:44.370202 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:44.372659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:44.372659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:44.372659 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:44.372659 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 279/576] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:44.372659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.392713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e90b-f2c7-4da8-94a2-b55988ce64f9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.419361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.441599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.442654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.443175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.443691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.444553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] RBAC: Adding query filter params (name=network) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:44.445123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.446751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.456699 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 298/577] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:44.457998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.500184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307a1b-ea13-484a-828a-efc795b48339 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.537911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.538284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.538515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.538798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.539393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.540968 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:44.550978 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:44.552097 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:44.553086 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:44.557209 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 280/578] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:44.558095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.581268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d72ff0-19d6-4345-8a74-db0088619a4d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.593087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.635048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.636359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.636877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.637492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.639323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:44.639903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.641630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.663139 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 299/579] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:44.663139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.682987 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2141-5853-4de6-9913-fe60817ee610 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.687931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.721554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.722172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.722453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.722836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.723403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] RBAC: Adding query filter params (type=network) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:44.723756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.725517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.732428 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 281/580] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:44.733492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.792807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920bc9-70ef-4222-82b2-2069e44fb31c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.835719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.836347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.836626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.836897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.837895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:44.838027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.839414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.850112 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 300/581] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:44.850789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.857458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b8c6d-7dc4-4a69-82d5-9c8d8b61227c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.879681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:44.900733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:44.902581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:44.903141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:44.903921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:44.913236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:44.916118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:44.933440 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:44.934474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:44.935432 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:44.937482 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 282/582] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:44.938533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:44.976283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ee20b-b90c-46ed-88d9-3780b444acc3 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:44.977525 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.020601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.020923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.021205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.021526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.022306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] RBAC: Adding query filter params (interface=public, service_id=fc6b17d77bf44023afad3826c5affb41, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:45.022660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.024108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.036955 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 301/583] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/endpoints?service_id=fc6b17d77bf44023afad3826c5affb41&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.038402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:45.085171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ad27-28ef-4c76-96d8-bb971a926f16 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.125756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.126746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.127607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.128291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.129460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:45.130014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.132722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.141890 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 283/584] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.143223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:45.172396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.185636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066a3b6-62db-44de-b89c-141ea863d8e2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.236937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.237643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.237913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.238192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.238899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:45.239211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.240791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36ad9-bd69-4979-afd0-7c364c1a1daa None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.254865 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 302/585] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.256517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acaee8-5cf1-4f57-8699-5ab949ae19da None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.256813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acaee8-5cf1-4f57-8699-5ab949ae19da None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.257086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acaee8-5cf1-4f57-8699-5ab949ae19da None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.264912 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 303/586] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:45.264912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acaee8-5cf1-4f57-8699-5ab949ae19da None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:45.304319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.326980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f044-4f08-4680-be7a-7f891e58da8d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.377542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.379123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.379661 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.380125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.381720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.384250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.403218 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acaee8-5cf1-4f57-8699-5ab949ae19da None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.429577 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 284/587] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:49:45 2022] GET /identity/v3/services => generated 772 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.431168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:45.444372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.445379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.445851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.446322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.447661 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] RBAC: Adding query filter params (interface=public, service_id=338aa53e8d4f4f90a82504e1a5c4fdbb, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:45.448436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.451032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca1f91-61c7-4589-ac9b-6ba2144a0b4b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.470267 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 304/588] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:49:45 2022] GET /identity/v3/endpoints?service_id=338aa53e8d4f4f90a82504e1a5c4fdbb&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.470570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.471251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.471251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.500619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:45.502532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:45.517512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.585696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c5a7d-4b72-4926-9020-5f9b71e365e6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.594614 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 305/589] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:45.595786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:45.644730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.645283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.645572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.645853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.646641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=eacd66576ed2425a8ad1fe0b69a4c5eb) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:45.647479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.648986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade45d6-abee-4bbb-ade6-8eb022ebf8fa None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.656917 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 285/590] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:45 2022] GET /identity/v3/role_assignments?user.id=eacd66576ed2425a8ad1fe0b69a4c5eb&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:45.657329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e49ca-94b8-48a8-968c-c6ada634657f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.658167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e49ca-94b8-48a8-968c-c6ada634657f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.658167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e49ca-94b8-48a8-968c-c6ada634657f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.659257 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 286/591] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:45.665012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.665354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.665576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.689965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:45.691657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:45.706896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.773251 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 287/592] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:49:45.798576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8979616-6dcf-4e20-b2dd-341d24b8249f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:45.859551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:45.860246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:45.860544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:45.860819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:45.861453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:45.862935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:45.871952 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 306/593] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:49:45 2022] GET /identity/v3/services => generated 772 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:46.016886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.164097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e0e48-d095-4326-b573-4075dc0da7bd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:46.170204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.183910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:46.208899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:46.209682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:46.209961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:46.210232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:46.224175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:46.230523 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:46.231532 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:46.232859 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 288/594] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:46.238985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.252530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:46.318076 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e0854-c157-4435-9632-e070abda86e6 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:46.362288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:46.363232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:46.363654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:46.364040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:46.379146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:46.389350 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:46.390427 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:46.391553 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 307/595] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:46.399754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.476447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd735bbf-591d-4008-a1e9-50f0744d9b7a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:46.542824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:46.573346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:46.573889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:46.574170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:46.574444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:46.575118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:46.575420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:46.576843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:46.588061 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 289/596] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:46.595439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.613401 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e79b-95e5-4354-abcc-0f65c99eabd3 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:46.694808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:46.702594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:46.703243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:46.703637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:46.704046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:46.705244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:46.705560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:46.707161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:46.734859 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 308/597] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:46.738206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1849-3196-4923-a548-0c70d9bb9c07 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:46.742501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:46.842939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:46.844034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:46.844503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:46.844956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:46.890763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:46.893513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:46.924926 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:46.925962 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:46.926862 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:46.926862 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:46.926862 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:46.926862 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 290/598] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:46.937052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.023269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74007a-a6af-461f-ab29-a8814cc34e18 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.096251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.100494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.101169 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.101445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.101719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.123412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.125587 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:47.151817 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:47.154089 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:47.156331 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:47.156331 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:47.156331 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:47.156331 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 309/599] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:47.156854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06abd4-0062-4bfe-9f56-bec1c1401a6f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.164297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.194178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.221472 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.222092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.222369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.222643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.223315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:47.223639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.225082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.253213 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 291/600] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:47.259420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.367230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660577a0-81dd-41b4-9d73-85a3673bc954 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.406520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.407165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.407443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.407765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.415887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.417345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:47.431181 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:47.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:47.435759 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 292/601] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:47.441071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.460791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f64298-715c-4fce-ad49-8b86d66ef820 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.502218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.561327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.562339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.562843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.563329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.563844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ea3e-f945-45d5-8152-8b92c24a2be6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.564535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:47.565074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.566703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.597887 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 310/602] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:47.609023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.609654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.609933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.610206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.610954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:47.611260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.611366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.612741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.634740 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 293/603] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:47.640829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:47.742472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.747887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.750825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3338-91fc-49f8-acde-427aee99a24e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.788267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:47.789007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:47.789284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:47.789553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:47.863900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:47.866110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:47.902720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6667-5580-4cee-a938-cb10abf4905a None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:47.974258 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 294/604] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 30 00:49:47.974887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:47.980187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.012397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.013379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.013854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.014321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.028772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.031017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:48.057865 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:48.058872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:48.059872 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:48.070394 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 311/605] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:48.078936 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.088159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b6295-74d4-474d-8c8e-1491df72e77d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.129403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.130131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.130416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.130689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.151288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.152883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:48.186037 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:48.186940 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:48.186940 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:48.186940 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:48.190364 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 295/606] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:48.197447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.294500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:48.321731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eead-ece9-4d29-8ce9-d054b8273ba4 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.328733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:48.347011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecd21-f6b3-466a-aec8-45292bb4db7a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.385922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.386949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.387454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.387968 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.388839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:48.389444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.391157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.405601 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 296/607] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:48.411446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.431710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.433128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.440092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.441053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.442870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:48.444256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.447755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.506576 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 312/608] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:48.515888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.541532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432036-1066-48a6-a030-4f5fbafcd9b4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.566528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:48.617253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.617913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.618195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.618452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.635093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.636630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:48.661393 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:48.663259 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:48.665152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:48.674025 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 297/609] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:48.681553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:48.720712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9fa09-0e7d-40c9-ba0c-970445c420f2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.783305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.784289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.784762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.785207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.817908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:48.885574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8635-9ff9-46ae-90db-acdb170f7727 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:48.888104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.890835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305b660-ba6e-4bab-a1b1-22ff5310b54e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.928178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:48.930224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.930790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.931245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.932714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:48.933261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:48.935341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b60ab-87e8-424c-a7b9-02bba0e0a6c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:48.960617 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 298/610] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:48.960617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04b7bf-4ea7-455a-b8e9-089f06c02cec None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:48.960617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04b7bf-4ea7-455a-b8e9-089f06c02cec None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:48.960617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04b7bf-4ea7-455a-b8e9-089f06c02cec None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:48.964611 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 299/611] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:48.965771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04b7bf-4ea7-455a-b8e9-089f06c02cec None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.006857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:49.098942 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 313/612] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:49.098942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999127cb-2224-4b0c-a803-ab137ebd3752 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.098942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999127cb-2224-4b0c-a803-ab137ebd3752 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.098942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999127cb-2224-4b0c-a803-ab137ebd3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.104193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04b7bf-4ea7-455a-b8e9-089f06c02cec None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:49.121991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-999127cb-2224-4b0c-a803-ab137ebd3752 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:49.123182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-999127cb-2224-4b0c-a803-ab137ebd3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:49.150697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:49.151143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.151422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.151730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.152484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=d6a6265c254040d9a0d8ae45b29d3073) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:49.152826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:49.154249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fb4eb-9ecf-42c3-b42a-01279567d9d8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 300/613] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:48 2022] GET /identity/v3/role_assignments?user.id=d6a6265c254040d9a0d8ae45b29d3073&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac1e0-287a-467a-bc47-013ad45e1c38 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac1e0-287a-467a-bc47-013ad45e1c38 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac1e0-287a-467a-bc47-013ad45e1c38 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 301/614] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:49.170326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ac1e0-287a-467a-bc47-013ad45e1c38 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.231612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:49.260184 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 314/615] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:49.263133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f251-5671-48c3-868d-7b1b0428d945 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.263796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f251-5671-48c3-868d-7b1b0428d945 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.264312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f251-5671-48c3-868d-7b1b0428d945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.305433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:49.315259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ac1e0-287a-467a-bc47-013ad45e1c38 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:49.341300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a042f251-5671-48c3-868d-7b1b0428d945 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:49.343106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a042f251-5671-48c3-868d-7b1b0428d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:49.362054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:49.363214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.363811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.364317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.378736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:49.382231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:49.402581 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:49.403678 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:49.404664 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:49.404664 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:49.404664 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:49.405899 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 302/616] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:49.412100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.501240 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 315/617] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:49.503337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763cb7-dec1-494a-9e25-22f7c238f007 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.503870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763cb7-dec1-494a-9e25-22f7c238f007 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.504356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763cb7-dec1-494a-9e25-22f7c238f007 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.514978 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 316/618] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:49.517760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.518362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.518940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.546236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ad19-b7d6-41fb-b9e5-bc06eb4fead0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:49.550104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:49.551469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:49.589555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:49.590967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.591674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.592164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.593057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:49.593676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:49.595093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:49.610077 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 303/619] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:49.616335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.628889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:49.738901 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 317/620] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:49.828569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.843893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb08c9-79ff-410a-9ad1-08f918cce5ff None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:49.892895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:49.893785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:49.894095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:49.894354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:49.907127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:49.910401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:49.952282 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:49.954152 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:49.956014 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:49.967964 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 304/621] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:49.970172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:49.993919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.116325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b08f2-45ed-4180-b690-b0e882a89c2b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.146062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.181623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.182306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.182677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.182948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] PATH_INFO: `/v3/services/network` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.183545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.185081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:50.195643 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:50.197602 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.200510 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 318/622] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:49 2022] GET /identity/v3/services/network => generated 88 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:50.205043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.209673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc8f4c-69f3-4203-84f3-788d1e926e02 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.282830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.283920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.284410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.284900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.286251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:50.286836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.289606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.320294 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 305/623] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:50.320294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.339106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113d92-b92e-4111-8f8c-6ccd70a30853 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.356794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.381896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.382783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.383232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.383701 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.384292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.385714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:50.395349 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:50.398195 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:50.400029 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 319/624] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:50.401601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.483302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626147fd-e098-4e2c-b5ef-07568c42afee None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.515932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.528460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.529110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.529392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.529670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.530410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.531733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:50.547350 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:50.560067 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:50.561766 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 306/625] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:50.563363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.586895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881e92a-849d-4cda-ba0e-c81a2f2cb4f4 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.646959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.647552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.647906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.648196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.648850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:50.649186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.650592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.664613 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 320/626] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:50.664613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.727126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e04cdd-2be4-404c-aaad-cd81bff32339 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.735553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.777910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.778874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.779342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.779853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.780900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=27db563707034f9aa2e8df84704ff249) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:50.781428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.783897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.795412 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 307/627] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/role_assignments?user.id=27db563707034f9aa2e8df84704ff249&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:50.797067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:50.850117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edbd-6794-4636-8d9f-dd36b35330e8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:50.915552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:50.916506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:50.916969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:50.917436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:50.919146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:50.919343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:50.921847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:50.925328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:50.931313 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 321/628] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:50.932993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:51.034302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88d954-bc67-4c8d-bd05-67e05a8381c9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.116217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.116822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.117254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.117677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.118750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:51.119225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.120815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6826f-1fa3-40ac-b295-fa45bda1bf66 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.123004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.132232 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 308/629] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:51.133086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:51.133636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:51.178583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.179415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.179895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.180412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.182845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] RBAC: Adding query filter params (type=network) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:51.183037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.186097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.198858 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 322/630] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:49:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:51.200249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:51.348404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34068cc-94ef-47f6-9a23-f19564608983 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.385854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:51.392207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207610a-9ac4-4635-a39c-432007743acc None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.422687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.424517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.425486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.426727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.428901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:51.429700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.433244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.442252 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 309/631] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:51.444404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:51.458525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.459619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.460141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.460613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.461778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:51.462298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.465193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.483113 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 323/632] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Oct 30 00:49:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:51.485448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:51.555998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:51.659420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa4cb8-c7be-4ded-9907-3716fae06f68 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.701526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf2e03-f2cd-4b32-81cc-5d2cb7afbc4e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:51.715244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.715942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.716226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.716506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.717236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.718703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f918cf5-b41d-475f-b043-9cbad819d9b0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.756827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:51.757573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.757972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.758327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.759108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:51.760684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:51.861172 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 310/633] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:51.863549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6fc4-0550-4107-9cc2-ea9d429ba006 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.864075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6fc4-0550-4107-9cc2-ea9d429ba006 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.864625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6fc4-0550-4107-9cc2-ea9d429ba006 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.868439 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 311/634] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:51.878852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:51.879504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:51.880018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:51.898824 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 324/635] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:51.934744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:51.937050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:52.018994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:52.136362 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 312/636] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:52.466605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:52.599099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:52.613000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb143-38c8-4474-9177-58c722352bbb None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:52.654015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:52.655006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:52.655487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:52.655950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:52.656712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:52.658270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:52.670065 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:52.671233 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:52.672168 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 325/637] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:52.685056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:52.833706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67e282-0c15-44bd-baaf-22de2da15487 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:52.878929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:52.879573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:52.879879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:52.880161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:52.880941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:52.881248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:52.882741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff208bb-4b32-4443-92cb-081eec19bee0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:52.888709 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 313/638] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:52.896620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:53.021822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467567-b82d-403d-833c-fff7fc96024f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:53.071476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:53.072492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:53.072930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:53.073382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:53.074476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:53.074931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:53.076546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff936da5-5897-4812-b5b4-ce57ede39b9b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:53.084260 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 326/639] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:53.578927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3d6b-08fc-4521-aff9-b2783961baeb None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:53.579300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3d6b-08fc-4521-aff9-b2783961baeb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:53.579480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3d6b-08fc-4521-aff9-b2783961baeb None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:53.581117 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 314/640] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:53.587073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5d34-e0c0-4530-857a-1a46da650249 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:53.587567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5d34-e0c0-4530-857a-1a46da650249 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:53.587688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5d34-e0c0-4530-857a-1a46da650249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:53.606264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-49ce5d34-e0c0-4530-857a-1a46da650249 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:53.607475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-49ce5d34-e0c0-4530-857a-1a46da650249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:53.622273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:53.679737 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 327/641] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:53.919685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782b7d-cff2-48c6-bfbe-81fdbec8bc98 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:53.920176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782b7d-cff2-48c6-bfbe-81fdbec8bc98 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:53.920176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782b7d-cff2-48c6-bfbe-81fdbec8bc98 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:53.921704 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 315/642] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:53.926919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:53.927931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:53.928211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:53.947047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:53.948254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:53.952571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4782b7d-cff2-48c6-bfbe-81fdbec8bc98 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:53.982820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:54.027154 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 328/643] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:54.069963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4782b7d-cff2-48c6-bfbe-81fdbec8bc98 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.110063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.111055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.111633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.112124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.124600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:54.129914 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:54.131548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:49:54.132443 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:54.135095 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 316/644] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:49:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:54.140762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:54.228123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:54.250282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b8002-e9f8-47e8-89b6-048278025482 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.286140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.287306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.287618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.287893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.288572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:54.288877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:54.290327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:54.303458 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 329/645] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Oct 30 00:49:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:54.309068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:54.312101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:54.413962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:54.421343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b3c8-1987-4c06-959d-49ec640b258f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.428527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d146-eeaf-4713-9a0a-ed72fc3aeb13 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.462511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.463141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.463422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.463726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.464388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:54.465943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84a577-7ec3-4d37-bae0-7d10cc2c3a86 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:54.469383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.470233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.470688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.471192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.483849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:54.485533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:54.502837 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:54.506675 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:54.507707 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:54.507707 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:54.507707 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:54.507707 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 317/646] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:54.509621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:54.518992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:54.524575 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 330/647] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:49:54 2022] POST /identity/v3/services => generated 269 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:54.641052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf001721-a3fb-45e7-a3cb-08d532699ebd None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.685345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703c4db-ba01-442b-b3f4-539f5521506a None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.685871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703c4db-ba01-442b-b3f4-539f5521506a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.685871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703c4db-ba01-442b-b3f4-539f5521506a None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.687241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.687576 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 331/648] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:54.687914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.688199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.688474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.689211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:54.689522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:54.691014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:54.693463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.693753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.694090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:54.695083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.706806 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 318/649] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:54.713807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:54.718955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:54.720325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:54.806110 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 332/650] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:54 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:54.845353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f22dd-392c-42ea-abf4-aa6f69ee3065 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:54.881703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:54.882338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:54.882615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:54.882886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:54.890459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:54.891961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:49:54.904693 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:54.905684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:54.907238 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 319/651] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:49:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:54.911422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.015961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f44f1-937c-42d8-86e4-db7f26a8bd2d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.050789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.056556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.057121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.057657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.057929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.058675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:55.058985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.060450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.079950 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 333/652] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Oct 30 00:49:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:55.089047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.106252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.153255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.205297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27c13-0789-450c-a299-c5be640a895b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.218073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8138-6ead-40df-ace3-bac375f32390 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.246506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.247470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.247783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.248062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.260570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.260987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.261635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.261914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.262189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.262781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.263484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.264164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:55.275786 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:55.276863 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:49:55.277787 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 334/653] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:55.284258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:55.285650 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:55.286604 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:55.287498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:55.287498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:55.287498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:55.287498 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 320/654] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:55.293582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.354795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.418159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9030-dc66-48ed-bf7e-2b355028e86c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.426387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225939f-5ecc-4b28-886c-80cedeaf6c57 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.438007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.461437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.462055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.462337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.462797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.463242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:55.463542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.465006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8570e-b36d-43f0-97cd-13ae02569862 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.468702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.469353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.469640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.475155 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.475379 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 335/655] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:55.475477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:55.475477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.475676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a773-0443-4c4f-bcbf-0a11fbc91190 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.475676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a773-0443-4c4f-bcbf-0a11fbc91190 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.475676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a773-0443-4c4f-bcbf-0a11fbc91190 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.475969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc53045-330a-4b2e-a5f9-8816b4667027 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.476061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 336/656] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:55.477927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a773-0443-4c4f-bcbf-0a11fbc91190 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.490550 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 321/657] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:55.495454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.495846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.496271 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.515540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:55.516783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:55.593431 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 322/658] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:55 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:55.595077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.618815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a773-0443-4c4f-bcbf-0a11fbc91190 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.639229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.660415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.661397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.661861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.662353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.663489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:55.664489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.666461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.678470 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 337/659] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:55.743198 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.747792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946ea78-9245-4ba9-9428-cd522567060f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.787960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.788618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.788895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.789169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.796887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.798392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:49:55.812346 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:55.813383 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:55.814297 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 323/660] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:55.819913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.910575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.934310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acc69f-4c15-43ab-bac2-cc64b4821978 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.976526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.977645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.978424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.978974 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.980412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:55.981188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.983149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.002062 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 338/661] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:49:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:56.008902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:56.035392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01cc96-5e47-4975-9c90-420d6820fc24 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:56.059424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:56.073413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:56.074209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.074594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.074986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.075787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:56.077266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df2066-8d15-4495-abe3-57bf1058127a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.128377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2650b-afa1-438c-9e58-2ab32acd59a7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:56.159739 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 324/662] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:49:55 2022] POST /identity/v3/services => generated 241 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:56.181158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:56.181771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.182054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.182332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/27db563707034f9aa2e8df84704ff249/roles/a918d4af36e84052b12c698f6c33185f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.184922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=27db563707034f9aa2e8df84704ff249, role_id=a918d4af36e84052b12c698f6c33185f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:56.186434 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78cbde-b985-4e87-b732-4cde77c40bb3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.290284 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 339/663] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:49:56 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/27db563707034f9aa2e8df84704ff249/roles/a918d4af36e84052b12c698f6c33185f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:56.820026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b4a-e979-4f1e-9a79-0efa33857afe None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.820737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b4a-e979-4f1e-9a79-0efa33857afe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.821244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b4a-e979-4f1e-9a79-0efa33857afe None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.823058 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 325/664] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:56.828482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.828930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.829098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.847744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:56.849481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:56.924898 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 340/665] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:56 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:49:57.198402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289b4a-e979-4f1e-9a79-0efa33857afe None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.221117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.311856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289b4a-e979-4f1e-9a79-0efa33857afe None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.347304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.347976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.348253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.348526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.349652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.351024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:57.360138 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:57.366890 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:57.368801 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 326/666] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:57.370691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.403692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.493808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6936a-12b7-4406-9e7b-7ad3cbd9459d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.530758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.531444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.531752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.532026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.532666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:57.532966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.534329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.540152 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 341/667] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:57.548714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.641128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.660741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1f2e-0f87-4cf7-b815-b8f2b3718252 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.698333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.699191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.699191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.699191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.699698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:57.700965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.703607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.706946 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 327/668] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:57.713226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.811462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.821928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556877b6-3ec2-4dfc-813c-0fe58d670a01 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.859259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.860699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.861176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.861661 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.862578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] RBAC: Adding query filter params (interface=public, service_id=bac16a730a56409bbdcf6ce1d03da8a8, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:57.863084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.864763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c1268-6116-40cd-8bd9-3154b9e2f7e1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.875640 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 342/669] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/endpoints?service_id=bac16a730a56409bbdcf6ce1d03da8a8&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:57.877284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.982681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a76f9-a91a-4916-8004-a114410c28c2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.018875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.020309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.021411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.021889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.022742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.024398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a112f-3034-460a-9dc5-154ffb6106a9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.035775 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 328/670] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/services => generated 1258 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:58.157321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3238-ab93-462d-9115-28243316a98f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.158006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3238-ab93-462d-9115-28243316a98f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.158506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3238-ab93-462d-9115-28243316a98f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.160374 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 343/671] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:58.169885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ae424-6e29-4776-83c0-92d3ccd7c4ce None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.170548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ae424-6e29-4776-83c0-92d3ccd7c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.171079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ae424-6e29-4776-83c0-92d3ccd7c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.192223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-447ae424-6e29-4776-83c0-92d3ccd7c4ce None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:58.194011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-447ae424-6e29-4776-83c0-92d3ccd7c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.197616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:58.271709 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 329/672] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:58.413228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4c2c1-8f94-4e9d-aff1-d83c535c5efd None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.413530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4c2c1-8f94-4e9d-aff1-d83c535c5efd None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.413717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4c2c1-8f94-4e9d-aff1-d83c535c5efd None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.415289 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 344/673] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:58.420821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587cf94-c817-45ab-8aba-b1bdebb4e876 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.421108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587cf94-c817-45ab-8aba-b1bdebb4e876 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.421388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587cf94-c817-45ab-8aba-b1bdebb4e876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.440231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7587cf94-c817-45ab-8aba-b1bdebb4e876 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:58.441433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7587cf94-c817-45ab-8aba-b1bdebb4e876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.513416 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 330/674] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:49:58.557235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4c2c1-8f94-4e9d-aff1-d83c535c5efd None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.645045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc7cb-f585-4d25-a1d3-5fee1f432f01 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.646317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc7cb-f585-4d25-a1d3-5fee1f432f01 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.646849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc7cb-f585-4d25-a1d3-5fee1f432f01 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.649785 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 331/675] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:49:58.654137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.654677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.655147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.671373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4c2c1-8f94-4e9d-aff1-d83c535c5efd None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.679280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:58.681047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.711326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.711964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.712247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.712520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.713167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.714740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.754701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:58.760350 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 332/676] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:49:58.787663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.794025 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 345/677] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:58.910651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdd8d8-fcb9-4cb6-907b-8300816635c1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.956980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.957985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.958913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.959367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.960192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.961877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:58.972129 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:58.973415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:49:58.974498 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:58.977057 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 333/678] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:58.983770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.006210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.049383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.102158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b644d5-646e-4ab0-9dee-c4a1e3ef8257 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.126741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.137235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.137999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.138283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.138553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.139193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.139493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.140914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.147183 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 346/679] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:59.152811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.177780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7663-a009-498b-893a-5989c594c4be None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.222583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.223517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.224002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.224422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.238482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:49:59.246277 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:59.247491 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:49:59.248684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:59.250599 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 334/680] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:49:59.256855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.264595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.308450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168531-e0e1-4f9c-a395-78a0b7152c9e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.350551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.351910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.353132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.353705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.354821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.355352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.358080 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.370723 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 347/681] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:59.371964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.401487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f375f-851c-42cd-bfb5-7d1dc550262d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.433416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.449267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.450800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.451241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.451989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.452846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.453307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.455141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.469976 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 335/682] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:59.475463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.509055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff968f-92c1-4823-8645-af26bacbd4fd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.556299 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.557359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.557844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.558940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.559913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] RBAC: Adding query filter params (interface=public, service_id=40fa4267140c4406b113d4624e61891a, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.560833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.563130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.571027 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 348/683] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/endpoints?service_id=40fa4267140c4406b113d4624e61891a&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:59.577117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.584826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783015c3-fe11-4d47-a0b6-a4dc74f1bec4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.591388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.626994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.627678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.627957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.628229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.641288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.642712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:49:59.662157 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:49:59.663197 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:49:59.664131 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:49:59.664131 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:49:59.664131 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:49:59.668193 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 336/684] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:49:59.669218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.709381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.734126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9649505-c169-4fea-bb47-b715510c5306 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.784503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fd27-dcef-4583-994d-f2950c4fb491 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.786943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.787858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.788354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.788830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.789682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.791348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.799158 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 349/685] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/services => generated 1497 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:49:59.826435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.827060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.827344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.827638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.828304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.828611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.830032 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.845766 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 337/686] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:59.852028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.939189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.977714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094152b5-f5e2-47d2-84f6-c5077588979d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.019240 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.020002 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.020370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.020738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.031291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.033010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:50:00.046622 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:00.047822 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:50:00.048966 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:00.051289 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 350/687] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:00.057776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.159887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.167535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbec8-ca05-4e5d-bf53-71f14f1228df None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.205002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.205623 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.205905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.206178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.206918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:00.207222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.209253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d1524-6ce2-408c-b484-7dd677f14d07 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.235870 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 338/688] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:00.237445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.351453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51109f-30ef-4cfc-a169-63e6da93a351 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.388391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.394110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.394737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.395017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.395316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.467520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.469027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.476350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca70d-032c-4719-8efa-19a0623184d1 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.476855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca70d-032c-4719-8efa-19a0623184d1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.476954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca70d-032c-4719-8efa-19a0623184d1 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.478234 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 339/689] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:00.484178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.484458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.484730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.510684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:00.512066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.605672 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 351/690] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:50:00.605903 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 340/691] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:00.612907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.645035 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.721015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205833-d259-4b81-a669-eb383bdf78b2 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.770475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.771123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.771407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.771700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.783931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.785417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:00.802163 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:00.803423 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:00.804855 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:00.804855 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:00.804855 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:00.804855 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 352/692] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:00.809000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.918521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.939719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.974265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0452-f50f-4df9-8cae-5fa54469bf86 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.027929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.028978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.030072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.030072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.031318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:01.031971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.034498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.063971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.065813 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 341/693] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:01.070690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.102658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f9fe2-c535-4c7b-a2fd-58307622e987 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.160535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.162444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.163112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.163792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.165127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.168120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-e27a3cdc-09b3-4c76-b7f9-d39ca3eb5487 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:01.185915 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:01.187746 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:01.189416 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 353/694] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:01.190912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807b44-0010-4801-8da8-9ea40a7da59d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.190912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807b44-0010-4801-8da8-9ea40a7da59d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.191164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807b44-0010-4801-8da8-9ea40a7da59d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.193742 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 354/695] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:01.198185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807b44-0010-4801-8da8-9ea40a7da59d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.250467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12e1f-1add-4257-b039-3f49c3cf1f71 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.289054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.289700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.289979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.290269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.297812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.299270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-20e93112-e5bc-4017-bd2e-c17d3c9225a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:50:01.312170 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:01.329622 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 342/696] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.330864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.331966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.331966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.349401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:01.353775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.404048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807b44-0010-4801-8da8-9ea40a7da59d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.442984 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 343/697] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:01.444215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.470263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.471472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.472072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.472612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.474449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:01.475061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.477959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.491647 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 355/698] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:01.495183 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.516761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.655005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecebb24-ee97-458c-b60c-afe8c7bad52d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.683846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e780-2319-442d-9805-f63469c0038d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.732259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.741252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.742630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.743352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.744049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.745984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:01.752760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.752760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.753281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.753281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.753281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.753881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.755466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:01.756053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.758284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.767296 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 356/699] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:01.781470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.782971 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 344/700] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:01.787888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.921128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.962773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fa28-cd94-418f-a52c-439017bb0e7f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.964759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90310ae-2401-4d1d-ae17-b180c71a36db None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.027639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.028747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.028910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.029310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.030178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] RBAC: Adding query filter params (role.id=a918d4af36e84052b12c698f6c33185f, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=27db563707034f9aa2e8df84704ff249) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:02.030696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.032533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:02.037341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.038598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.039084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.039567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.047996 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 345/701] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/role_assignments?user.id=27db563707034f9aa2e8df84704ff249&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=a918d4af36e84052b12c698f6c33185f => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:02.048280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.048413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:02.048586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ccd6a-2e9c-4cba-82e8-b402def14fd9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:02.157685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:02.164175 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 357/702] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:02.184436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be99e-5e18-4659-930f-e601ad159c1e None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.184945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be99e-5e18-4659-930f-e601ad159c1e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.185430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be99e-5e18-4659-930f-e601ad159c1e None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.188009 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 358/703] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:02.198070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.198563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.199053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.233427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:02.234983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:02.248354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731d21-4204-4dd0-ad1a-d5f576f671fa None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.325422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.325422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.325422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.325422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.325838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.325838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:02.338291 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:02.339358 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:02.340275 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:02.342334 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 346/704] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:02.349414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:02.395382 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 359/705] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:02.485264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcef7bf-c1b4-49d9-be15-b0ed04d990ab None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.505892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:02.534859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.536429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.537225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.537760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.538639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:02.539154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.540839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:02.546665 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 347/706] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:02.553312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:02.636006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:02.714259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53a24f-fa38-4d47-80cd-232ce28c2eac None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.778516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.779572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.780101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.780607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.781794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:02.782344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.785023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:02.788340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:02.797765 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 360/707] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:02.801513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:02.946557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b794-ee02-4102-8854-be7fbee65d83 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.958222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ff2f-44c4-4286-b333-cf3043fe869d None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:02.993701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:02.994724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:02.995097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:02.995444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:02.996159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:02.997627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:03.007746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:03.012640 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:03.013691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:03.014552 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:03.020572 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 348/708] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:50:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:03.021560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:03.033621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:03.034668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:03.035137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:03.035675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:03.036592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] RBAC: Adding query filter params (interface=public, service_id=69120fc9c82742df88ec7189ae79db95, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:03.037092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:03.040274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:03.053395 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 361/709] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:50:02 2022] GET /identity/v3/endpoints?service_id=69120fc9c82742df88ec7189ae79db95&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:03.054399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:03.117752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:03.218639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5291-e6f1-4679-8030-3c228d26bedf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:03.260561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501d74-236c-40fa-8f8b-7935224fd084 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:03.295216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:03.297064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:03.297776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:03.298469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:03.299859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:03.300604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:03.304736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:03.316315 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 349/710] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:03.323210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:03.335310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:03.336582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:03.337144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:03.337826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:03.339009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:03.341633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:03.364647 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 362/711] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:50:03 2022] GET /identity/v3/services => generated 1497 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:03.526674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d39f2-f3ff-451c-9e84-947a5bf67d1b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:03.545444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:03.592524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:03.593502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:03.593965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:03.594430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:03.595527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:03.596086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:03.598761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173dfc-3d5e-440b-a94f-5b620840e03c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:03.608639 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 350/712] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:03.620762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:03.724760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:03.816375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79b20e-7aac-47d1-ac10-1b5addcdc146 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:03.880401 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:03.881404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:03.881961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:03.882449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:03.883676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:03.886249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b7492-4bfb-4f31-baa5-fcee886f44a7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:04.010075 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 363/713] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:04.575438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fee4-3ae6-4b69-b011-cdacc66ea306 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:04.576053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fee4-3ae6-4b69-b011-cdacc66ea306 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:04.576053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fee4-3ae6-4b69-b011-cdacc66ea306 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:04.577672 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 351/714] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:04.583389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:04.583928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:04.584007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:04.604421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:04.605662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:04.690098 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 364/715] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:04.962563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7fee4-3ae6-4b69-b011-cdacc66ea306 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:05.080724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7fee4-3ae6-4b69-b011-cdacc66ea306 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:05.119964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:05.120615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.120897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.121173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.121763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:05.123143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6c7b7e30-c71e-46cb-93cf-d613dcf3ce73 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:05.133499 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:05.134566 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:05.135896 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 352/716] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:05.141029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:05.145664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9748e-e02b-447f-bec8-7d925d54320c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.145984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9748e-e02b-447f-bec8-7d925d54320c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.146230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9748e-e02b-447f-bec8-7d925d54320c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.147894 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 353/717] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:05.155066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.155775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.156206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.179501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:05.181095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:05.182048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:05.216056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:05.252152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7018b46-2c4e-469d-81e2-02af874c136c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:05.260603 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 354/718] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:05.291217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:05.291904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.292193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.292473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.293158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:05.293469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:05.294938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:05.301517 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 365/719] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:05.307281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:05.425540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c721d-c4c2-4262-80f5-3be0d5623096 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:05.467856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:05.468417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.468699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.468974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.469643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:05.469948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:05.471382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:05.477380 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 355/720] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:05.619220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:05.746990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:05.751450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be50d4-21cb-43a8-9949-7ddd475881a3 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:05.812543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:05.813642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.814150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.814654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] PATH_INFO: `/v3/services/image` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.815790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:05.818492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:05.836252 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:05.837308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:05.838141 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:05.839497 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 366/721] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Oct 30 00:50:05 2022] GET /identity/v3/services/image => generated 86 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:05.845780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:05.881182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:05.959072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484501-0983-497f-b031-e2e3913a13c1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:05.996842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:05.997517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:05.997751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:05.998027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:05.998675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] RBAC: Adding query filter params (name=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:05.998985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:06.000397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3342ad-3547-4842-beab-1f4bfa2e9182 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:06.006137 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 356/722] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:06.011958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:06.124425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc73c2-2231-4129-a7ad-3d2421f55133 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:06.163930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:06.164564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:06.164851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:06.165148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:06.165783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] RBAC: Adding query filter params (type=image) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:06.166147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:06.167539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948e48c-6518-4975-b48f-e403bfd8d96d None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:06.175805 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 367/723] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:06.430425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ff99f-4e33-4506-87b4-127c03225f61 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:06.430688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ff99f-4e33-4506-87b4-127c03225f61 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:06.431028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ff99f-4e33-4506-87b4-127c03225f61 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:06.432825 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 357/724] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:06.438920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd318-5d39-4a70-8047-c0500acfbc7c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:06.439329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd318-5d39-4a70-8047-c0500acfbc7c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:06.439478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd318-5d39-4a70-8047-c0500acfbc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:06.468051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d6dfd318-5d39-4a70-8047-c0500acfbc7c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:06.470446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d6dfd318-5d39-4a70-8047-c0500acfbc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:06.512706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:06.590776 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 368/725] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:06.838234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99e4e-e3fd-42a4-88af-7efaeb36159c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:06.838670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99e4e-e3fd-42a4-88af-7efaeb36159c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:06.838775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99e4e-e3fd-42a4-88af-7efaeb36159c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:06.840623 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 358/726] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:06.846469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:06.846985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:06.847060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:06.866760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:06.868017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:06.947267 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 369/727] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:07.096573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99e4e-e3fd-42a4-88af-7efaeb36159c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.244939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:07.253057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99e4e-e3fd-42a4-88af-7efaeb36159c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.272534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.325761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.327320 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.328080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.328777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.330266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.333882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:07.352745 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:07.354931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:50:07.356893 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:07.367842 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 359/728] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:07.372482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.376769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:07.397307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501842bc-4908-4338-8027-a3e36913cc4c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.447517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.448217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.448512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.448795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.449419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.452144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:07.463119 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:07.464308 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:07.465448 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 370/729] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:07.471069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.595408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025b677-d94a-4f3d-97c7-294a4fd2e3b2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.631175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51ce83-a60a-41a0-abba-f17e4927eb2c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.648086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.649622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.650501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.651013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.652216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:07.652784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.655797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.678271 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 360/730] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:07.684249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.689107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.690258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.690764 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.691262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.692728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:07.693270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.694958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.700979 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 371/731] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:07.706490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.724992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:07.737436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:07.864808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993ae7-9ae1-4e0d-9aac-e9b1c1c6e5d8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.878143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171c22b-44d6-4795-a1f8-f9dc4cfcf015 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:07.903147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.903721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.904013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.904290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.904935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:07.905248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.906723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8a9a1-6429-487a-856b-e5a91ee440b4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 372/732] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59bf05-ec6d-41cd-821c-c565cc4a03e9 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59bf05-ec6d-41cd-821c-c565cc4a03e9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59bf05-ec6d-41cd-821c-c565cc4a03e9 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 373/733] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:07.919644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59bf05-ec6d-41cd-821c-c565cc4a03e9 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:07.941921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:07.942912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.943382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.943913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.945055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:07.945606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:07.948194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212544c8-f1a5-4058-b9b2-4779ada2ba1d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:07.956945 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 361/734] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:07.959190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:07.959832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:07.960160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:07.988172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:07.990305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:08.030225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:08.058839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59bf05-ec6d-41cd-821c-c565cc4a03e9 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:08.075150 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 362/735] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:08.077001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:08.113165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:08.114778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.115328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.115896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.117206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] RBAC: Adding query filter params (interface=admin, service_id=40fa4267140c4406b113d4624e61891a, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:08.117809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:08.120100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:08.120859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:08.129817 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 374/736] 10.222.0.26 () {56 vars in 1249 bytes} [Sun Oct 30 00:50:07 2022] GET /identity/v3/endpoints?service_id=40fa4267140c4406b113d4624e61891a&interface=admin®ion_id=RegionOne => generated 193 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:08.138247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:08.280106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1251564-e407-45f7-b82e-26ed904f1e39 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:08.343735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ae2e3-38db-4370-ab7b-4e90fbf2fc5a None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:08.347191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:08.348639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.349407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.350251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.351911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:08.354920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb74fc-2196-4a00-9b6a-02e70dcb4be8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:08.396121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:08.396750 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.397034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.397313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.397932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:08.399316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:08.406078 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 375/737] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:50:08 2022] GET /identity/v3/services => generated 1497 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:08.406676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:08.417544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:08.461513 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 363/738] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:08.550356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3878c90-66ae-4822-b907-a4633789798c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:08.575248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e95e9-0263-410a-8824-446816555421 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.575738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e95e9-0263-410a-8824-446816555421 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.576201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e95e9-0263-410a-8824-446816555421 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.585160 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 364/739] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:50:08.587958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6aa3-2524-4878-b750-5c8ad160b7a8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.588420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6aa3-2524-4878-b750-5c8ad160b7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.588896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6aa3-2524-4878-b750-5c8ad160b7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.605607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:08.606243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:08.606527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:08.606807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:08.607488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:08.608985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:08.619999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d25b6aa3-2524-4878-b750-5c8ad160b7a8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:08.622006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d25b6aa3-2524-4878-b750-5c8ad160b7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:08.683465 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 376/740] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:50:08 2022] POST /identity/v3/services => generated 242 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:08.749255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:08.763624 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 365/741] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:09.249761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:09.366242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:09.442190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea87cb-0229-4cf4-a428-ac3079b16fe7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:09.505407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:09.506237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:09.506665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:09.507204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:09.508443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:09.510994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a94606-1b6a-40c1-a653-4d97fe60dbce None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:09.609684 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 377/742] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:50:09 2022] POST /identity/v3/services => generated 242 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:10.805960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f122d-31c6-4b1f-9685-989ecfc855c6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:10.806993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f122d-31c6-4b1f-9685-989ecfc855c6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:10.807492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f122d-31c6-4b1f-9685-989ecfc855c6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:10.809509 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 366/743] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:10.815900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67157b0-f955-4eda-9be8-9105e19c46ca None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:10.817476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67157b0-f955-4eda-9be8-9105e19c46ca None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:10.817956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67157b0-f955-4eda-9be8-9105e19c46ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:10.839077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b67157b0-f955-4eda-9be8-9105e19c46ca None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:10.840694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b67157b0-f955-4eda-9be8-9105e19c46ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:10.929516 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 378/744] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:10 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:10.996977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5777b-74c1-4d54-aaa2-6d580171f9b8 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:10.997621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5777b-74c1-4d54-aaa2-6d580171f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:10.998164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5777b-74c1-4d54-aaa2-6d580171f9b8 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.000211 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 367/745] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:11.006089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.006412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.006654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.027768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:11.029043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:11.080496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:11.110402 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 379/746] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:11 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:11.300238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5777b-74c1-4d54-aaa2-6d580171f9b8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.391601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.458590 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5777b-74c1-4d54-aaa2-6d580171f9b8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:11.496861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:11.497600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.497879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.498155 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.498764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:11.500189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:11.510553 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:11.512907 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:11.514922 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 368/747] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:11.522634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.583120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428303b2-7828-48fe-bd17-92755a1c0ab7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:11.603182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:11.647402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6593f-250a-4573-8ad6-bed43f0bf96c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:11.654150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:11.655172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.655767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.657368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.658511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:11.661518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:11.680286 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:11.682160 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:11.683777 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:11.688546 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 380/748] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:11.694696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.699217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:11.699894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.700177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.700455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.701115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:11.701423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:11.702900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:11.714511 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 369/749] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:11.717377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.765895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:11.836012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f6cdc-4a81-46c6-a88c-c9c33e145c6b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:11.856855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e21c-6781-4da1-9bcd-63248baeb7c4 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:11.865536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:11.875507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:11.876458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.876977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.877280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.878028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:11.878404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:11.880468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:11.891304 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 370/750] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:11.892481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:11.914859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:11.916345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:11.916846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:11.917346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:11.918219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] RBAC: Adding query filter params (name=compute) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:11.918720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:11.921189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:11.930283 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 381/751] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:11.936605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:12.017676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfe1e-522f-4c56-9dc5-2c6843d32042 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:12.058496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:12.059528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.060070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.060566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.061483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:12.063098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14204ea-949a-4ea4-92b9-252ad2c3e94b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:12.105789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e9cbc-02ff-49dd-bf86-78c5cb4ffdca None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:12.114407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:12.145487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:12.146237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.146623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.146979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.147822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:12.148251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:12.149842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b8b1d-767a-4bf7-b40f-36f8c5273fa3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:12.155312 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 382/752] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:12.156599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52efdca-f0b1-4836-96c6-3f5279933eb2 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.156934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52efdca-f0b1-4836-96c6-3f5279933eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.157187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52efdca-f0b1-4836-96c6-3f5279933eb2 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.159624 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 383/753] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:12.162609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52efdca-f0b1-4836-96c6-3f5279933eb2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:12.179516 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 371/754] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:12.182144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.183344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.183865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.218137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:12.220121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:12.324368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:12.359505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52efdca-f0b1-4836-96c6-3f5279933eb2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:12.367646 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 372/755] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:12.426440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:12.427537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.428210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.428584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.429694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] RBAC: Adding query filter params (interface=public, service_id=32c627600360486d8d040cef461b6440, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:12.430148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:12.432768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d93fb7-51f1-46b8-8a11-52864569ae42 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:12.444731 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 384/756] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:50:12 2022] GET /identity/v3/endpoints?service_id=32c627600360486d8d040cef461b6440&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:12.454366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:12.626449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c209df-c4e1-44e4-a275-ad07d6ba1a66 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.627267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c209df-c4e1-44e4-a275-ad07d6ba1a66 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.627995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c209df-c4e1-44e4-a275-ad07d6ba1a66 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.630051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58bb4a-43ab-4d68-bce4-287bac1c0e04 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:12.631864 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 385/757] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:12.642646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.643986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.645163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.670339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:12.671639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:12.686164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:12.687319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.687886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.688049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.688551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:12.690671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:12.707625 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 373/758] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:50:12 2022] GET /identity/v3/services => generated 1957 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:12.707625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:12.708627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:12.811713 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 386/759] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:12.820006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:12.883516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e7af-14e3-4a34-9b14-3d28367e2760 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:12.926068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:12.927089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:12.927578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:12.928179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:12.929353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:12.931064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:12.934118 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING keystone.server.flask.common [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 30 00:50:13.002712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:13.002712 dv-cinder-master-3-smb devstack@keystone.service[81635]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:50:13.002712 dv-cinder-master-3-smb devstack@keystone.service[81635]: [parameters: {'id': '844a4d23a80d4143813a70a2bd2eec3e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 30 00:50:13.002712 dv-cinder-master-3-smb devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 844a4d23a80d4143813a70a2bd2eec3e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 844a4d23a80d4143813a70a2bd2eec3e. Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:13.010142 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:50:13.011742 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Oct 30 00:50:13.013821 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:50:13.016230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:50:13.019809 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:13.022510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'id': '844a4d23a80d4143813a70a2bd2eec3e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.025318 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:13.029689 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 844a4d23a80d4143813a70a2bd2eec3e. Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.034022 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 374/760] 10.222.0.26 () {60 vars in 1140 bytes} [Sun Oct 30 00:50:12 2022] POST /identity/v3/projects => generated 262 bytes in 312 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:13.036214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.213537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.230915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410d90e-b89e-4e53-8d10-c198232484d3 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:13.295094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:13.296302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:13.297724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:13.298225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:13.315735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:13.319869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:50:13.347260 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:13.352194 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:13.353130 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:13.355703 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 387/761] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:13.361848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:13.362157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.367037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e06a8e-8a69-4256-926b-6991c2032fcf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:13.412197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:13.413270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:13.413778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:13.414114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:13.414534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] PATH_INFO: `/v3/services/image` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:13.415975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:13.418209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:13.432781 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:13.434709 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:13.436442 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:13.442761 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 375/762] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/services/image => generated 86 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:13.453754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.555398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceb066-eccf-46e3-80d6-5ec45e396174 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:13.585154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b229-e6b1-49ea-8c91-aee3746d8652 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:13.617549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:13.618927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:13.619217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:13.619691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:13.620683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:13.621070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:13.623198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:13.632021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:13.632810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:13.633193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:13.633589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:13.634362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] RBAC: Adding query filter params (name=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:13.634778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:13.636365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:13.651138 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 376/763] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:13.652615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.654127 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 388/764] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:13.708077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:13.813706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc94b2-f999-4c32-81d1-4bea36ca550d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:13.877995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:13.878783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:13.879240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:13.879748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:13.880867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] RBAC: Adding query filter params (type=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:13.881402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:13.883535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:13.890567 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 377/765] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:13.899546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:13.995753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:14.016242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d53f-5741-4e9c-a38a-37f26a43ab40 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:14.062766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502693-57d7-4155-a786-248a876468be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:14.063402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502693-57d7-4155-a786-248a876468be None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:14.063695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502693-57d7-4155-a786-248a876468be None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:14.064027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502693-57d7-4155-a786-248a876468be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:14.064711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46502693-57d7-4155-a786-248a876468be None admin] RBAC: Adding query filter params (interface=public, service_id=a24128e6d7e443bf94caac0a73e2b2f0, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:14.065026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46502693-57d7-4155-a786-248a876468be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:14.066471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46502693-57d7-4155-a786-248a876468be None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:14.073441 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 389/766] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:50:13 2022] GET /identity/v3/endpoints?service_id=a24128e6d7e443bf94caac0a73e2b2f0&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:14.079075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:14.195893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:14.214778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198038eb-a333-4c7f-9a1f-fe064251734e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:14.273303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:14.274441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:14.275048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:14.275537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:14.276601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:14.278380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030ba0f-4a2d-4515-bbde-d9247dcb69d9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:14.286971 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 378/767] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:50:14 2022] GET /identity/v3/services => generated 1957 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:14.727738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2a7e-2b7b-4cd4-8587-3f5dbb079a2f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:14.728005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2a7e-2b7b-4cd4-8587-3f5dbb079a2f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:14.728340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2a7e-2b7b-4cd4-8587-3f5dbb079a2f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:14.730135 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 390/768] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:14.736341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29407-b460-48ef-953d-12258fcc3e52 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:14.736645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29407-b460-48ef-953d-12258fcc3e52 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:14.736967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29407-b460-48ef-953d-12258fcc3e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:14.759397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3bf29407-b460-48ef-953d-12258fcc3e52 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:14.761105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3bf29407-b460-48ef-953d-12258fcc3e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:14.881822 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 379/769] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:14 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:15.161251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2a7e-2b7b-4cd4-8587-3f5dbb079a2f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:15.231951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3bd-60e4-4f6f-b391-3adb20cd97b0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:15.232756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3bd-60e4-4f6f-b391-3adb20cd97b0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:15.232756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3bd-60e4-4f6f-b391-3adb20cd97b0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:15.234191 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 380/770] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:15.240216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:15.240502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:15.240820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:15.267469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:15.269466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:15.328503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2a7e-2b7b-4cd4-8587-3f5dbb079a2f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:15.371010 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 381/771] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:15.379637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:15.379637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:15.379637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:15.379637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:15.380066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:15.380066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:15.382033 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 391/772] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:15.387816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:15.407867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:15.497498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f88d1-baef-4af2-9bf2-e4998497295e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:15.535630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:15.536250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:15.536533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:15.536811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:15.537477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:15.538912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:15.615991 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 382/773] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:50:15 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:15.660741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:15.678365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:15.789899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1f06-a9bd-4e4b-a071-781cda46c547 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:15.829431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:15.830085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:15.830365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:15.830641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:15.831800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:15.833194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:15.842559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:15.843799 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:15.845316 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 392/774] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:15.850330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:15.928161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:15.962477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856afa5-010c-49c8-9d9d-5223777bf52d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:16.001244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:16.001860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.002147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.002401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.003039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:16.003933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:16.005306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:16.014317 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 383/775] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:16.020286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:16.106385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:16.132158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00066954-00b5-4daf-8f9f-9f4cb013fab0 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:16.171043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:16.171727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.172081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.172364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.173062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:16.173399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:16.174892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfba73-5f47-4eb7-bea2-05759540edc5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:16.181993 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 393/776] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Oct 30 00:50:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:16.188477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:16.301612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ee137-c28f-4560-9d43-536026d35a29 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:16.333102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:16.339896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:16.340762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.341194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.341616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.342458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:16.344040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644142e-d7ec-4432-95bc-ce63268f4c39 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:16.356710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0171cd-a214-43c0-9630-7ddab8632232 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.356950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0171cd-a214-43c0-9630-7ddab8632232 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.357207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0171cd-a214-43c0-9630-7ddab8632232 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.358716 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 394/777] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:16.364389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce990c-6613-4ca8-a1b7-d9fcb91a5ed8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.364665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce990c-6613-4ca8-a1b7-d9fcb91a5ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.364945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce990c-6613-4ca8-a1b7-d9fcb91a5ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.384761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-59ce990c-6613-4ca8-a1b7-d9fcb91a5ed8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:16.385971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-59ce990c-6613-4ca8-a1b7-d9fcb91a5ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:16.450572 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 384/778] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:16.483447 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 395/779] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:16.713128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc4b3-df69-4b6d-b1b1-d0a5025b6a91 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.713478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc4b3-df69-4b6d-b1b1-d0a5025b6a91 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.713688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc4b3-df69-4b6d-b1b1-d0a5025b6a91 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.715369 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 385/780] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:16.721054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.721346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.721624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.741243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:16.742458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:16.772925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc4b3-df69-4b6d-b1b1-d0a5025b6a91 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:16.828525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 396/781] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:16.829096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:16.905536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc4b3-df69-4b6d-b1b1-d0a5025b6a91 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:16.943093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:16.943657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:16.943944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:16.944225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:16.944883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:16.946387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:16.989283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:50:16.989283 dv-cinder-master-3-smb devstack@keystone.service[81635]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 30 00:50:16.989283 dv-cinder-master-3-smb devstack@keystone.service[81635]: [parameters: {'user_id': '47fdd37376984affb95fac93caf2fc42', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 30 00:50:16.989283 dv-cinder-master-3-smb devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:16.997011 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:16.997931 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:50:16.998936 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:50:16.999902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:17.000824 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:17.001711 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'user_id': '47fdd37376984affb95fac93caf2fc42', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:17.002550 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 30 00:50:17.005439 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.007672 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 386/782] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Oct 30 00:50:16 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:17.008868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.122034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827e80-e972-4b2e-83f9-822fc164f685 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.124691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.161433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.162474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.162979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.163467 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.171911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.174225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.181379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:50:17.189006 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:17.194749 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:17.196021 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 397/783] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:17.198340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.250010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fa3e1-772c-413e-a24b-3b3f3eeb7d71 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.273347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:17.289785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.290831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.291357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.291975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] PATH_INFO: `/v3/services/image` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.292894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.294713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:17.304902 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:17.306416 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:50:17.307279 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:17.313705 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 387/784] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:17.317008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.326423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9ec0a-056c-4f70-920b-ce27a6e70857 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.385017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.386048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.386561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.387331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.388315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:17.389391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.391395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.438102 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 398/785] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:17.473419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86046-750b-4ff6-b327-8b2fcda83a1a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.491569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:17.515979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.517008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.517487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.517969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.519460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] RBAC: Adding query filter params (name=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:17.520029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.521683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.530936 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 388/786] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:17.540696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.688733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94bacc-b9f3-412f-856c-a4020a189187 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.709060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:17.732407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.732932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.733211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.733477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.734121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:17.734427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.735841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0aa10-4952-43d4-8715-1f7c467408d0 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.741886 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 399/787] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:17.752006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.871170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602450-d032-4869-b53d-8ba211b1793f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.908986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.909673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.909951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.910254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.910910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.912469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bcd36-9fc3-4c81-9715-e6b55a156982 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.008854 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 389/788] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:18.091065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbeb7f-60b4-4ab9-abcf-ad0d435aa02f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.091389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbeb7f-60b4-4ab9-abcf-ad0d435aa02f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.091750 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbeb7f-60b4-4ab9-abcf-ad0d435aa02f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.093336 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 400/789] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:18.100122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.100469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.100729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.122756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:18.124469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.165872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:18.242457 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 390/790] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:18.559888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbeb7f-60b4-4ab9-abcf-ad0d435aa02f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.684315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbeb7f-60b4-4ab9-abcf-ad0d435aa02f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.725005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.725816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.726241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.726653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.728027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.729744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a55378-c6a5-4645-a33f-b91a9411dd17 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.731745 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 401/791] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:18.738338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.894700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0d36-5e0f-46b5-a948-1719564c894c None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.895067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0d36-5e0f-46b5-a948-1719564c894c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.895258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0d36-5e0f-46b5-a948-1719564c894c None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.896803 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 402/792] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:18.902034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.902323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.902599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.921873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350965-5169-4e7e-9d1b-897747ecf0ac None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.923888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:18.925145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.968725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.969530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.969938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.970348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.971257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.973679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.975782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.004729 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 403/793] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:19.041979 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 391/794] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/users => generated 285 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:19.279784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.403816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3ae37-1aa2-457b-9b48-4a07cb0f4e6d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.444810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.445450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.445735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.446008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.458550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:19.463518 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:19.465165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:50:19.467471 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:19.469623 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 404/795] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:19.470900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.493637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.595766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794fc2-88cb-4275-b18c-6a8bb913d717 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.638733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.639374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.639711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.639964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.640668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:19.641053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.642470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.659649 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 392/796] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:19.659941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.711213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.785666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d95e-a676-4811-b43c-256c2e8ed51a None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.824950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.825756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.826163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.826570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.837892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.839543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:19.856038 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:19.857845 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:19.859722 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:19.859722 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:19.859722 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:19.859722 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 405/797] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:19.864219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.070800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:20.095018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a765022-f114-4140-90c7-0cdb992ea374 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.134926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.135460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.135848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.136082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.136722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:20.137094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.138410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.152351 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 393/798] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:20.158294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.260911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:20.275502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f8711-3acd-4497-9404-e4e54264b5dd None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.314561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.315389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.315706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.316005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.324842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.326390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-dc8f14ae-9acf-4aeb-9b4e-147d300b2dbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:50:20.339649 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:20.341417 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:50:20.344165 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:20.345994 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 406/799] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:20.347628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.476088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482d28-2b78-4204-9720-83c5efd2e5d0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.522479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.523109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.523390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.523704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.524484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:20.524791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.526245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.534666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c97b-894d-430e-8128-71410c351578 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.534956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c97b-894d-430e-8128-71410c351578 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.535267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c97b-894d-430e-8128-71410c351578 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.536943 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 407/800] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:20.541395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:20.545214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.545758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.546220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.557721 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 394/801] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:20.559196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.574033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:20.575706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:20.652622 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 408/802] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:20.714192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549bb9e-9521-40ec-8907-b90748f309fa None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.758918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.759489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.759802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.760164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.772043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.773487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:20.788670 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:20.789666 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:20.791191 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:20.791191 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:20.791191 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:20.791191 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 395/803] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:20.795513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.880311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:20.902765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0645f-4f79-47a7-b23f-bc6f1da2b2d5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.940370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.940984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.941269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.941521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.942168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:20.942458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.943871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.948854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.956698 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 409/804] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:20.962160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:21.006571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:21.070689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d4801-6a1b-4d4b-820e-6acc0781323c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.080569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bb595-e56f-4f08-ae92-bff5acb2e7f5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.110411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.111512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.112369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.112830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.113790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.115537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.117016 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.117918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.118194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.118462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.125904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.127344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.129350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:50:21.140565 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:21.142621 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:21.144157 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 410/805] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:21.149336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:21.149336 dv-cinder-master-3-smb devstack@keystone.service[81635]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:50:21.149336 dv-cinder-master-3-smb devstack@keystone.service[81635]: [parameters: {'id': 'fc9bbc354a9148f8afd47fa494e8f764', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:50:21.149336 dv-cinder-master-3-smb devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:50:21.149692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:21.158548 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:50:21.160140 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:50:21.161111 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:50:21.161996 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:50:21.163010 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:50:21.163840 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'id': 'fc9bbc354a9148f8afd47fa494e8f764', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:50:21.164780 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:21.165685 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 396/806] 10.222.0.26 () {60 vars in 1137 bytes} [Sun Oct 30 00:50:20 2022] POST /identity/v3/domains => generated 190 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:21.166588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:21.293338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d97575-0fe9-4348-ba34-496940b4e73e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.301103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666759-db5a-4edb-a5c2-4f3f132b2555 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.316741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:21.342708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.344038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.344330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.344604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.345311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:21.345649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.347102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.351503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.352629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.353720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.353720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.360827 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 411/807] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:21.366950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:21.368687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.370284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:21.386066 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:21.387361 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:21.388531 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:21.388531 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:21.388531 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:21.388531 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 397/808] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:21.395904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:21.445333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:21.512529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818b81c-6604-46d9-99bb-02a8b5614e02 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.530915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed6c8-1c22-4da3-81de-f75013311fa5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:21.563739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.564758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.565472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.565753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/a918d4af36e84052b12c698f6c33185f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.568749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=eacd66576ed2425a8ad1fe0b69a4c5eb, role_id=a918d4af36e84052b12c698f6c33185f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.570205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.572121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:21.572745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.573028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.573309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.573965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:21.574273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:21.575673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fd40b-059b-41bb-945d-4d4fed4b3a2e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:21.588331 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 398/809] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:21.589626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641dbd-eb8e-4f3b-ac6e-fb55fe43e955 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.589909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641dbd-eb8e-4f3b-ac6e-fb55fe43e955 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.590181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641dbd-eb8e-4f3b-ac6e-fb55fe43e955 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.591436 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 399/810] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:21.598044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:21.598271 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:21.598485 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:21.620044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:21.621301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:21.636718 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 412/811] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:50:21 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/eacd66576ed2425a8ad1fe0b69a4c5eb/roles/a918d4af36e84052b12c698f6c33185f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:21.662327 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:21.735650 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 400/812] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:22.007067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:22.123781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2a7df-ba58-4539-b799-2c5f9caf57a8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:22.145890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:22.165516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:22.166166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:22.166446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:22.166722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:22.178973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:22.183986 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:22.185379 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:22.186815 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 413/813] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:22.191528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:22.311388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b0d9d-7470-49cb-825d-5cc6ccf7c792 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:22.355022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:22.355681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:22.356047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:22.356323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:22.357034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:22.357339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:22.358830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:22.369789 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 401/814] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:22.376929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:22.506650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e2089-244c-4dbd-b076-1844faad5302 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:22.544519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:22.545165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:22.545444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:22.545729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] PATH_INFO: `/v3/users/7ccb3886eb26499aa16969a59e9cf30e` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:22.563940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] RBAC: Authorizing `identity:get_user(user_id=7ccb3886eb26499aa16969a59e9cf30e)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:22.566334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:22.568471 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 414/815] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:22 2022] GET /identity/v3/users/7ccb3886eb26499aa16969a59e9cf30e => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:22.575301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:22.598239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:22.712368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbba89-dcf8-4614-b610-795f03421855 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:22.753387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:22.754159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:22.754526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:22.754926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:22.756254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] RBAC: Authorizing `identity:get_project(project_id=aa921b71779849199a04b2cff86880b8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:22.757805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ae7e0-58f0-4dfa-a0cf-313f1cf2f867 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:22.759617 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 402/816] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:22 2022] GET /identity/v3/projects/aa921b71779849199a04b2cff86880b8 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:22.765757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:22.805988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:22.881800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f40bd-888f-4aef-94ac-f65f811680b7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:22.934179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:22.935280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:22.935802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:22.936333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8/users/7ccb3886eb26499aa16969a59e9cf30e/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:22.939650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa921b71779849199a04b2cff86880b8, user_id=7ccb3886eb26499aa16969a59e9cf30e, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:22.942917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e8283-28fc-4a89-a300-e495d2f26271 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:23.047033 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 415/817] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:50:22 2022] PUT /identity/v3/projects/aa921b71779849199a04b2cff86880b8/users/7ccb3886eb26499aa16969a59e9cf30e/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:23.894032 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0446c9-5352-4e36-a4e4-daca688a12cc None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:23.894801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0446c9-5352-4e36-a4e4-daca688a12cc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:23.894801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0446c9-5352-4e36-a4e4-daca688a12cc None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:23.895937 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 403/818] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:50:23.904413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:23.904890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:23.905285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:23.925388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:23.926594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:24.004901 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 416/819] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:24.279079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0446c9-5352-4e36-a4e4-daca688a12cc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:24.432020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:24.460732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0446c9-5352-4e36-a4e4-daca688a12cc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:24.519426 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:24.520405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:24.520823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:24.521259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:24.538901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:24.547449 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:24.548641 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:24.550474 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 404/820] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Oct 30 00:50:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:24.557041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:24.586310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:24.695052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fabb19-9e0f-4051-b1e7-37e597827add None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:24.744203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:24.745281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:24.745772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:24.746273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:24.747608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:24.748163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:24.750798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:24.766889 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 417/821] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Oct 30 00:50:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:24.774132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:24.887933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a254813-b78b-4575-b70e-1a780d504154 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:24.926086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:24.926801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:24.926984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:24.927260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:24.938286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:24.939829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:24.942662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:24.956603 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:24.972540 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:24.974723 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:24.974723 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:24.974723 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:24.974723 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 405/822] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:24.975227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.098990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.113268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b26f-08bc-4ccf-8cc1-6dc316ddc193 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.153249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.153767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.154052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.154327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.154985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:25.155292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.156721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e1eb8-42b9-4f06-b34d-23600a642d74 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.174068 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 418/823] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:25.175419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.232232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6a5f-8013-48fa-af27-8145d0d4ccad None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.232517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6a5f-8013-48fa-af27-8145d0d4ccad None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.232843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6a5f-8013-48fa-af27-8145d0d4ccad None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.234177 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 419/824] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:25.239840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.240129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.240408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.259800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:25.261014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:25.285438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033c67-9a46-4cd6-8e91-49abd2b3c1f5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.320556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.321192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.321470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.321744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.329994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.332053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.336252 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 420/825] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:50:25.343995 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:25.345144 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:25.346331 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 406/826] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:25.350833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.464273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.490906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653a353-4c93-496c-8b9a-90010500cc1f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.533493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.534130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.534410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.534676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.535432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:25.535770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.537248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.558012 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 421/827] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:25.564304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.612293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.653367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.706210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc9cd2-dd43-4a30-8d94-fdbdbebde0f3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.734691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c42be-0244-4eb6-9545-8132d4549ef9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.756456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.757506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.758019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.758521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.773116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.773773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.774057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.774334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.777607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.787844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:25.797037 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:25.798905 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:25.801150 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 422/828] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:25.806253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.860366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.861891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.922846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.941369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e606d72-5958-4748-88f8-82b747e5967c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.980981 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 407/829] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:50:25.988422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.990676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.991612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.992071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.992777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.993610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:25.993983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.996403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.016583 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 423/830] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:26.018966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.099102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.119545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f570-7d66-4d0d-9b33-c7b324cf17aa None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.160733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.161402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.161681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.161957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.174806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.176038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0489-52e7-4a41-8c7d-305c285261b5 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.176368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:50:26.200956 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:26.202387 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:50:26.203342 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:50:26.203342 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:50:26.203342 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:26.203554 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 408/831] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:26.209635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.219080 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.219854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.220196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.220531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] PATH_INFO: `/v3/users/7ccb3886eb26499aa16969a59e9cf30e` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.222547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] RBAC: Authorizing `identity:get_user(user_id=7ccb3886eb26499aa16969a59e9cf30e)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.223226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.224697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.226482 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 424/832] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/users/7ccb3886eb26499aa16969a59e9cf30e => generated 285 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:26.232539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.349562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a91f0-3fa6-42a6-8e08-470c8c93348c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.361954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8f00-74c6-44cc-9836-ed10a0bb94c7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.415836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.426529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.426935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.427126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.427417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.427734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.427923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.428161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.428450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:26.428626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.428777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.430128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.444695 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 409/833] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:26.450935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.536179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.577583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cf65-31cf-4b73-bda2-3f9957515506 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.627699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.628323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.628667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.629243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.641046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.644223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.649151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.651123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:50:26.657891 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:26.659708 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:26.661074 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 410/834] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:26.667706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.791074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6243-b6cf-42dc-af58-c0a7b5bfcb92 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.791727 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 425/835] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 30 00:50:26.800752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.828642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.833125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.846578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.847836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.848334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.848826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.850123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:26.850663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.856070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.881865 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 411/836] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:26.883517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.912166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a79e-9e97-4cef-83a3-507e2c8b66a8 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.954673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.955337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.955646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.955926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.957121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] RBAC: Authorizing `identity:get_project(project_id=aa921b71779849199a04b2cff86880b8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.958587 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.960341 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 426/837] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/projects/aa921b71779849199a04b2cff86880b8 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:26.966622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.017819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662105-445a-4e33-a94c-94f14c048200 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.069838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:27.078005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.082914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.082914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.082914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.083322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40042a5d-ac18-401b-bcf6-b594f5be8df7 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.083456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] RBAC: Adding query filter params (role.id=a918d4af36e84052b12c698f6c33185f, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=eacd66576ed2425a8ad1fe0b69a4c5eb) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:27.083456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.084037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81fc19-d72e-4f5b-aec7-35d6286527e3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.102017 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 412/838] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/role_assignments?user.id=eacd66576ed2425a8ad1fe0b69a4c5eb&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=a918d4af36e84052b12c698f6c33185f => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:27.129459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.130099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.130378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.130654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.131420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=aa921b71779849199a04b2cff86880b8, user.id=7ccb3886eb26499aa16969a59e9cf30e) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:27.131762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.133461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bfa21-4dc7-43bf-989b-aadd353fe1b8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.140659 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 427/839] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:50:26 2022] GET /identity/v3/role_assignments?user.id=7ccb3886eb26499aa16969a59e9cf30e&scope.project.id=aa921b71779849199a04b2cff86880b8&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:29.427168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e698-2593-4270-bdbd-2bdde8b4acc6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:29.428028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e698-2593-4270-bdbd-2bdde8b4acc6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:29.428217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e698-2593-4270-bdbd-2bdde8b4acc6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:29.429603 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 413/840] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:29.435676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:29.436178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:29.436268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:29.457006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:29.458320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:29.537592 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 428/841] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:29.808551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e698-2593-4270-bdbd-2bdde8b4acc6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:29.885634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:29.969956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e698-2593-4270-bdbd-2bdde8b4acc6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:30.028367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:30.029438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:30.029931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:30.030420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:30.032358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:30.035027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b749b-e07b-44c8-bf37-7b8821fd315f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:30.037966 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 414/842] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:30.043981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:30.142483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:30.155338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956e690-35f9-4e49-9451-eb9effd9beda None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:30.191937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:30.192667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:30.193034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:30.193396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:30.194211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:30.195785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4be24-b5f2-46c7-89c5-388c74ff1297 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:30.297501 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 429/843] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:50:30 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:32.616320 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b0a1-d8e7-4818-9480-cc935b9ca2da None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:32.617256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b0a1-d8e7-4818-9480-cc935b9ca2da None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:32.617256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b0a1-d8e7-4818-9480-cc935b9ca2da None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:32.618468 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 415/844] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:50:32.624394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:32.624912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:32.625033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:32.645166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:32.646421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:32.732086 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 430/845] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:33.007573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4b0a1-d8e7-4818-9480-cc935b9ca2da None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:33.115660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4b0a1-d8e7-4818-9480-cc935b9ca2da None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:33.137087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:33.152299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:33.152986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.154783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.154783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] PATH_INFO: `/v3/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.155091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] RBAC: Authorizing `identity:get_role(role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:33.156552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:33.158542 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 416/846] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:33 2022] GET /identity/v3/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:33.164809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:33.261605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:33.303926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704295c-04c1-4653-8723-adcf05d8bfa2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:33.362041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:33.363121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.363630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.364119 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] PATH_INFO: `/v3/users/564331d484fe4df086405b8fd3e4f643` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.388535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] RBAC: Authorizing `identity:get_user(user_id=564331d484fe4df086405b8fd3e4f643)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:33.391303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:33.395153 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 431/847] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:33 2022] GET /identity/v3/users/564331d484fe4df086405b8fd3e4f643 => generated 286 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:33.401763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:33.514691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f0905-58a3-47d5-9e58-f7695e524991 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:33.553510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:33.554215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.554541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.554864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.556199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] RBAC: Authorizing `identity:get_project(project_id=aa921b71779849199a04b2cff86880b8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:33.557742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f0ea-7735-43d1-b0af-25bbadc81557 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:33.559596 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 417/848] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:33 2022] GET /identity/v3/projects/aa921b71779849199a04b2cff86880b8 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:33.565846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:33.705979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd009e4-2bf4-43b8-93f6-4edb02387b54 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:33.755558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:33.763096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:33.764263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.772160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.772160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8/users/564331d484fe4df086405b8fd3e4f643/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.772160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa921b71779849199a04b2cff86880b8, user_id=564331d484fe4df086405b8fd3e4f643, role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:33.772160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da946a-ab92-4cd6-81f4-c64c12431537 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:33.851764 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 432/849] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:50:33 2022] PUT /identity/v3/projects/aa921b71779849199a04b2cff86880b8/users/564331d484fe4df086405b8fd3e4f643/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:36.054918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee53ae-3ccb-4f6c-8002-a0ef3df0f513 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.055460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee53ae-3ccb-4f6c-8002-a0ef3df0f513 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.055460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee53ae-3ccb-4f6c-8002-a0ef3df0f513 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.056725 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 418/850] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:36.062495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.063016 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.063016 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.082025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:36.083233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:36.156504 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 433/851] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:36.435055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee53ae-3ccb-4f6c-8002-a0ef3df0f513 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:36.496379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:36.539743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee53ae-3ccb-4f6c-8002-a0ef3df0f513 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:36.574363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:36.574993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.575268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.575538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] PATH_INFO: `/v3/roles/b28dbbda4e65402b8caba7d1305fb822` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.576942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] RBAC: Authorizing `identity:get_role(role_id=b28dbbda4e65402b8caba7d1305fb822)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:36.578302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:36.579690 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 419/852] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:36 2022] GET /identity/v3/roles/b28dbbda4e65402b8caba7d1305fb822 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:36.585724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:36.688187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575884ca-09e4-400d-a80c-ce7cb368f132 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:36.710649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:36.722602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:36.723902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.723902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.723902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] PATH_INFO: `/v3/users/564331d484fe4df086405b8fd3e4f643` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.724894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] RBAC: Authorizing `identity:get_user(user_id=564331d484fe4df086405b8fd3e4f643)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:36.726329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:36.727952 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 434/853] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:36 2022] GET /identity/v3/users/564331d484fe4df086405b8fd3e4f643 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:36.734723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:36.833367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9bce0-4cef-4314-9a59-6aac5ced428d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:36.856250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:36.866907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:36.867506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.867821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.868135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.933863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:36.935308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:37.032909 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 420/854] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:50:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:50:37.039193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:37.137448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f269e7-2c5a-43e3-9825-73909159a73b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:37.174040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:37.174666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:37.174943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:37.175211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] PATH_INFO: `/v3/projects/aa921b71779849199a04b2cff86880b8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:37.176339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] RBAC: Authorizing `identity:get_project(project_id=aa921b71779849199a04b2cff86880b8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:37.177767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7a631-9c0d-48e4-ac2d-3b2c62bc31cc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:37.179382 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 435/855] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:37 2022] GET /identity/v3/projects/aa921b71779849199a04b2cff86880b8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:37.186029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:37.211203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:37.283842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850215c-e591-487b-b09b-f6f155f06715 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:37.319173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:37.319800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:37.320084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:37.320359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:37.321082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] RBAC: Adding query filter params (role.id=b28dbbda4e65402b8caba7d1305fb822, scope.project.id=aa921b71779849199a04b2cff86880b8, user.id=564331d484fe4df086405b8fd3e4f643) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:37.321392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:37.322873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eddf89-ad13-431c-8c59-62aa2ccf0555 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:37.328960 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 421/856] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:50:37 2022] GET /identity/v3/role_assignments?user.id=564331d484fe4df086405b8fd3e4f643&scope.project.id=aa921b71779849199a04b2cff86880b8&role.id=b28dbbda4e65402b8caba7d1305fb822 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:39.550287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147d5-a882-4645-bff1-58190626d5b0 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.550850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147d5-a882-4645-bff1-58190626d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.551290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147d5-a882-4645-bff1-58190626d5b0 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.553623 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 436/857] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:39.559511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.560052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.560182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.579268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:39.580544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:39.653715 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 422/858] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:39.925724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240147d5-a882-4645-bff1-58190626d5b0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:39.992138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:40.042998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240147d5-a882-4645-bff1-58190626d5b0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.080737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.081372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.081647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.081915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.083037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.084497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0ab5-92aa-4b64-9596-2a642fc04a97 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.086078 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 437/859] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:40.092152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.112092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:40.195767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c848a-5637-4127-9d7a-f273b2943f32 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.230250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.230863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.231141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.231413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.232107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.233543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ec77-ef54-4d75-a3c0-15561e5ceb81 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.326484 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 423/860] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:42.590344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a57030-edd6-4e10-9673-607861747048 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:42.590907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a57030-edd6-4e10-9673-607861747048 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:42.591074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a57030-edd6-4e10-9673-607861747048 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:42.592463 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 438/861] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:50:42.598359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:42.598867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:42.598985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:42.618780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:42.620125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:42.690948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:42.695782 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 424/862] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:42.970559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a57030-edd6-4e10-9673-607861747048 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:43.081279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a57030-edd6-4e10-9673-607861747048 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:43.108475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:43.118025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:43.118724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:43.119006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:43.119278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:43.120456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:43.121898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07100f83-4ed3-452b-9c6d-d3b77d3320b6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:43.123559 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 439/863] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:50:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:43.133956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:43.310023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8810-54ba-493d-8851-daa0357a13a6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:43.368690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:43.369733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:43.370214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:43.370700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:43.371931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:43.374674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2322b7-0c1a-4a06-81be-a4fbac0e45ba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:43.456034 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 425/864] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:50:43 2022] POST /identity/v3/users => generated 286 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:45.807772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2872a-3dcd-45f1-9dc5-3a6081ef077b None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:45.808683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2872a-3dcd-45f1-9dc5-3a6081ef077b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:45.808683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2872a-3dcd-45f1-9dc5-3a6081ef077b None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:45.809650 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 440/865] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:45.815647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:45.816487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:45.816654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:45.836744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:45.837981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:45.898927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:45.919705 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 426/866] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:46.191836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2872a-3dcd-45f1-9dc5-3a6081ef077b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:46.307480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2872a-3dcd-45f1-9dc5-3a6081ef077b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:46.346558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:46.347206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:46.347486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:46.347787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:46.360079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:46.365145 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:46.368979 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:50:46.370583 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 441/867] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:50:46.374078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:46.528558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bab07e-9ca1-4f51-87e7-f852536b8bcf None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:46.565788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:46.566417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:46.566694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:46.566967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:46.567673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:46.567977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:46.569359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:46.579447 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 427/868] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:46.586085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:46.650889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:46.703644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7022c-d8d3-43b3-be20-704bb9aa4800 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:46.739477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:46.740400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:46.740689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:46.740958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] PATH_INFO: `/v3/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:46.757181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] RBAC: Authorizing `identity:get_user(user_id=cb0d0e9c099e4cf4a9a73bb9ef9541dd)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:46.758644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:46.760419 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 442/869] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:46 2022] GET /identity/v3/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:46.767086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:46.824549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:46.894325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfa8b5-0316-413b-b4ee-90326025f024 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:46.931261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:46.931936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:46.932219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:46.932493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] PATH_INFO: `/v3/projects/0f1c7f1835b843079483f4df3ff65ee5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:46.933700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] RBAC: Authorizing `identity:get_project(project_id=0f1c7f1835b843079483f4df3ff65ee5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:46.935142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a91fb-8041-4af6-a52d-a9ff8910bd5b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:46.936854 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 428/870] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:46 2022] GET /identity/v3/projects/0f1c7f1835b843079483f4df3ff65ee5 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:46.943816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:47.055128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ed54-3d9a-48df-8353-aca6d8472c03 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:47.071797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:47.094842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:47.095785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:47.096376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:47.096699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] PATH_INFO: `/v3/projects/0f1c7f1835b843079483f4df3ff65ee5/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:47.098703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] RBAC: Authorizing `identity:create_grant(project_id=0f1c7f1835b843079483f4df3ff65ee5, user_id=cb0d0e9c099e4cf4a9a73bb9ef9541dd, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:47.100275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a9e8d-148b-45f7-81bd-caf066ee429f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:47.164203 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 443/871] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:50:46 2022] PUT /identity/v3/projects/0f1c7f1835b843079483f4df3ff65ee5/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:49.403332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71d79-37c2-4849-84d0-c52a867412dc None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:49.403940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71d79-37c2-4849-84d0-c52a867412dc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:49.403940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71d79-37c2-4849-84d0-c52a867412dc None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:49.405254 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 429/872] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:49.412193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:49.412709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:49.413085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:49.432995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:49.434243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:49.509570 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 444/873] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:49.779360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71d79-37c2-4849-84d0-c52a867412dc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:49.883757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71d79-37c2-4849-84d0-c52a867412dc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:49.901039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:49.924015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:49.924015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:49.924015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:49.924015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:49.931296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:50:49.936183 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:50:49.937458 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:50:49.938363 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 430/874] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:50:49.943677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:50.044064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0db9a0-24df-4cec-90bc-c3a5d2067fed None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:50.045620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:50.079424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:50.080068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:50.080345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:50.080616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:50.081291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:50.081616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:50.082988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:50.092134 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 445/875] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:50.102438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:50.215797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9817-c7ee-4c5f-be08-7184d565a2dc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:50.250862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:50.251510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:50.251825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:50.252105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] PATH_INFO: `/v3/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:50.253303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cb0d0e9c099e4cf4a9a73bb9ef9541dd)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:50.254756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:50.256452 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 431/876] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:50:50 2022] GET /identity/v3/users/cb0d0e9c099e4cf4a9a73bb9ef9541dd => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:50.263221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:50.371964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8538b-c6a3-4dee-be77-d7de61f25b25 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:50.407697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:50.408335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:50.408619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:50.408893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:50.445253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:50.487150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:50.488792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:50.601278 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 446/877] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:50:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:50:50.607494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:50.656425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:50.711992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced758a-4c71-43f5-958d-3d978711a4c0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:50.748522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:50.749075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:50.749357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:50.750145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] PATH_INFO: `/v3/projects/0f1c7f1835b843079483f4df3ff65ee5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:50.751280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] RBAC: Authorizing `identity:get_project(project_id=0f1c7f1835b843079483f4df3ff65ee5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:50.752748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c569c4-d889-409c-905b-fb6f4a4c9766 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:50.754398 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 432/878] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:50:50 2022] GET /identity/v3/projects/0f1c7f1835b843079483f4df3ff65ee5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:50.760712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:50.825275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:50.872384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57569-062b-44a2-aa4c-fea2eecb9e29 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:50.915756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:50.916363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:50.916647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:50.916918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:50.917666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=0f1c7f1835b843079483f4df3ff65ee5, user.id=cb0d0e9c099e4cf4a9a73bb9ef9541dd) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:50.917973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:50.919418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e2c0d-79b3-4a9d-984c-5565f514391f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:50.926019 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 447/879] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:50:50 2022] GET /identity/v3/role_assignments?user.id=cb0d0e9c099e4cf4a9a73bb9ef9541dd&scope.project.id=0f1c7f1835b843079483f4df3ff65ee5&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:54.206235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d2bc-573a-4066-a172-81f54b84d2fa None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.206834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d2bc-573a-4066-a172-81f54b84d2fa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.206834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d2bc-573a-4066-a172-81f54b84d2fa None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.208138 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 433/880] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:54.213986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341ec66-99bc-4159-90a0-3cf4dd17bf88 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.214496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341ec66-99bc-4159-90a0-3cf4dd17bf88 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.214610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341ec66-99bc-4159-90a0-3cf4dd17bf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.234108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f341ec66-99bc-4159-90a0-3cf4dd17bf88 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:54.235318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f341ec66-99bc-4159-90a0-3cf4dd17bf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.305954 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 448/881] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:50:54.574331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d2bc-573a-4066-a172-81f54b84d2fa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.695558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d2bc-573a-4066-a172-81f54b84d2fa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.711554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:54.735760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.738861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.738861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.738861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.738861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.739351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e43cd-59c8-494a-9526-d4d41eab520d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.809852 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 434/882] 10.222.0.26 () {60 vars in 1138 bytes} [Sun Oct 30 00:50:54 2022] POST /identity/v3/domains => generated 267 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:57.098136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5591a0-233e-42cf-9755-2a9406032b10 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.098697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5591a0-233e-42cf-9755-2a9406032b10 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.098697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5591a0-233e-42cf-9755-2a9406032b10 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.100205 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 449/883] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:50:57.106073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.106496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.106703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.125574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:57.126782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:57.197025 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 435/884] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:50:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:50:57.471389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5591a0-233e-42cf-9755-2a9406032b10 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.534747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:57.579645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5591a0-233e-42cf-9755-2a9406032b10 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.613959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.614845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.615127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.615398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] PATH_INFO: `/v3/domains/7f64f5636d9348539ce6fee37c8a87c4` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.626679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7f64f5636d9348539ce6fee37c8a87c4)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.628169 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c498449-94b2-4ca5-890d-decfce5e7565 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.629872 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 450/885] 10.222.0.26 () {56 vars in 1183 bytes} [Sun Oct 30 00:50:57 2022] GET /identity/v3/domains/7f64f5636d9348539ce6fee37c8a87c4 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:57.635962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.740014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dcc0a-8dc5-491c-88d6-a8a23e9e98d1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.741656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:57.778928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.779565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.779882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.780732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.781427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.782943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb8b87-27fa-4b9e-8f72-4c3ab95eed9d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.889708 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 436/886] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:50:57 2022] POST /identity/v3/projects => generated 361 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:00.163781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d77b-aa35-4274-924f-b51b854103cd None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.164365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d77b-aa35-4274-924f-b51b854103cd None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.164365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d77b-aa35-4274-924f-b51b854103cd None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.165671 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 451/887] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:51:00.172373 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.172895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.173062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.194665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:00.195950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:00.267562 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 437/888] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:51:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:51:00.543340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5d77b-aa35-4274-924f-b51b854103cd None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:00.652942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5d77b-aa35-4274-924f-b51b854103cd None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:00.671162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:00.688343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:00.689025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.689304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.689577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] PATH_INFO: `/v3/domains/7f64f5636d9348539ce6fee37c8a87c4` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.690695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7f64f5636d9348539ce6fee37c8a87c4)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:00.692200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585add7-a327-4c65-a413-e1a6ccbf2df6 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:00.693767 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 452/889] 10.222.0.26 () {56 vars in 1183 bytes} [Sun Oct 30 00:51:00 2022] GET /identity/v3/domains/7f64f5636d9348539ce6fee37c8a87c4 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:00.700714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:00.785129 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:00.801006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19974-eaff-42a9-ac91-c51b9dbdbf8e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:00.835523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:00.836160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.837272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.837272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.837571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:00.838786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04c63-f1da-4bdb-8e38-8f44b8325133 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:00.922817 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 438/890] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:51:00 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:03.223104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92a1a-fa84-4b64-8597-466062a7e1a6 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:03.223885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92a1a-fa84-4b64-8597-466062a7e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:03.223885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92a1a-fa84-4b64-8597-466062a7e1a6 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:03.225425 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 453/891] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:51:03.231932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:03.232559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:03.232559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:03.253987 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:03.255250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:03.328008 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 439/892] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:51:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:51:03.600439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92a1a-fa84-4b64-8597-466062a7e1a6 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:03.715785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92a1a-fa84-4b64-8597-466062a7e1a6 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:03.751078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:03.751745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:03.752029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:03.752304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:03.763256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:51:03.768072 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:51:03.769637 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:51:03.770535 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 454/893] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:51:03.775409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:03.844716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:03.877204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcbbba-253c-4e78-95d3-ffe4a96e7c88 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:03.919397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:03.920511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:03.921263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:03.921801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:03.922747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:03.923137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:03.925070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:03.934000 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 440/894] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:51:03.943032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:03.992966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:04.051862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bbe7-a9e3-4c77-9b3c-6ebd343316ad None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:04.089888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:04.090544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:04.090825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:04.091100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] PATH_INFO: `/v3/users/6832588675374d04aa34e1c670ddcc63` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:04.105953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=6832588675374d04aa34e1c670ddcc63)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:04.107425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:04.109259 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 455/895] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:51:03 2022] GET /identity/v3/users/6832588675374d04aa34e1c670ddcc63 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:04.118541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:04.225179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7b985-35bd-4dd2-9873-fde9edfd40ef None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:04.257108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:04.262634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:04.263283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:04.263564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:04.264455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] PATH_INFO: `/v3/projects/721286bae9d44d5881aecdd40ef81032` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:04.265662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] RBAC: Authorizing `identity:get_project(project_id=721286bae9d44d5881aecdd40ef81032)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:04.272458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6c4bd-4141-4be2-8637-29376b820d75 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:04.272458 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 441/896] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:51:04 2022] GET /identity/v3/projects/721286bae9d44d5881aecdd40ef81032 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:04.275197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:04.385613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cb8a-b479-4663-8905-2b5487449cf1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:04.388168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:04.422341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:04.423196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:04.423525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:04.423912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] PATH_INFO: `/v3/projects/721286bae9d44d5881aecdd40ef81032/users/6832588675374d04aa34e1c670ddcc63/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:04.426000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=721286bae9d44d5881aecdd40ef81032, user_id=6832588675374d04aa34e1c670ddcc63, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:04.427558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a6a5-8454-492c-8f76-063157d973f3 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:04.499039 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 456/897] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:51:04 2022] PUT /identity/v3/projects/721286bae9d44d5881aecdd40ef81032/users/6832588675374d04aa34e1c670ddcc63/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:06.873581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56d376-b9d5-4046-9995-f54d66ee7f8e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:06.874167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56d376-b9d5-4046-9995-f54d66ee7f8e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:06.874167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56d376-b9d5-4046-9995-f54d66ee7f8e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:06.875512 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 442/898] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:51:06.881231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:06.881726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:06.881864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:06.900579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:06.901784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:06.975320 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 457/899] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:51:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:51:07.249412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56d376-b9d5-4046-9995-f54d66ee7f8e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:07.358307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56d376-b9d5-4046-9995-f54d66ee7f8e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:07.396326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:07.397978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:07.397978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:07.397978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:07.410657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:51:07.415510 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:51:07.417415 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:51:07.419444 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 443/900] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:51:07.424636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:07.519340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:07.526055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43179-1096-444b-9ac1-aadac89e9640 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:07.561515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:07.562177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:07.562563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:07.563054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:07.563854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:07.564257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:07.565746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:07.574477 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 458/901] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:07.581746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:07.643291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:07.689003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7da3a5-79b1-4ff6-a047-52fad0d5b4eb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:07.728267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:07.728924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:07.729206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:07.729483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] PATH_INFO: `/v3/users/6832588675374d04aa34e1c670ddcc63` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:07.730712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=6832588675374d04aa34e1c670ddcc63)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:07.732751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:07.734473 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 444/902] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Oct 30 00:51:07 2022] GET /identity/v3/users/6832588675374d04aa34e1c670ddcc63 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:07.741379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:07.863909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb836-20ea-4371-b653-341e350abd1f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:07.900234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:07.901597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:07.901597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:07.901597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:07.930285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:07.999388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.002386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.119660 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 459/903] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:51:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:51:08.126458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:08.231298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6354a0-c88f-4f20-9a2f-c07f227d1ed6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:08.264388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:08.273202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:08.273853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.274134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.279033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] PATH_INFO: `/v3/projects/721286bae9d44d5881aecdd40ef81032` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.279033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] RBAC: Authorizing `identity:get_project(project_id=721286bae9d44d5881aecdd40ef81032)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.279033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a218389-d4c9-4b04-878d-7f41fbfc17b7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.279033 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 445/904] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Oct 30 00:51:08 2022] GET /identity/v3/projects/721286bae9d44d5881aecdd40ef81032 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:08.285792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:08.352907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:08.405950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000242a-7b1b-4ed4-a356-4635728bebbe None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:08.445835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:08.446454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.446736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.447013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.447783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=721286bae9d44d5881aecdd40ef81032, user.id=6832588675374d04aa34e1c670ddcc63) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:08.448093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.449546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04603ca7-6d18-4df4-b2cf-d827334d4423 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.456495 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 460/905] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:51:08 2022] GET /identity/v3/role_assignments?user.id=6832588675374d04aa34e1c670ddcc63&scope.project.id=721286bae9d44d5881aecdd40ef81032&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:18.298711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:18.453570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=40 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:23.059159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2fb0-e3b7-4b6c-b370-eba10fcf6d6c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:23.059743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2fb0-e3b7-4b6c-b370-eba10fcf6d6c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:23.059743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2fb0-e3b7-4b6c-b370-eba10fcf6d6c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:23.061004 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 446/906] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:23.066247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:23.067408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:23.067408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:23.085188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:23.086362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:23.151418 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 461/907] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:23.420114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2fb0-e3b7-4b6c-b370-eba10fcf6d6c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:23.523152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2fb0-e3b7-4b6c-b370-eba10fcf6d6c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:23.557043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:23.557673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:23.557944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:23.558212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:23.568422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:23.569870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:23.585186 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:23.586715 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:23.590966 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:23.590966 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:23.590966 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:23.590966 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 447/908] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:23.592612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:23.693334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d3056-2bf5-4159-afa5-9c606fdc2e71 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:23.722356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:23.728678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:23.729294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:23.729574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:23.729848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:23.730768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:23.731073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:23.732475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d353de5-be83-4fab-a793-8a49777b5c0b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:23.743837 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 462/909] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:23.749800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:23.802743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:23.848234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde02c4e-c941-40d7-b008-0cf072eabac4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:23.882384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:23.883081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:23.883369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:23.883675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:23.884353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:23.885796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91fd3e-2775-4d71-8ade-5234c3eac679 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:23.954004 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 448/910] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Oct 30 00:52:23 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:52:26.138870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b053df-a100-4cea-b436-98e64a0d1fa3 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.139424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b053df-a100-4cea-b436-98e64a0d1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.139424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b053df-a100-4cea-b436-98e64a0d1fa3 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.140824 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 463/911] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:26.146792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.151109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.151109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.165359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:26.166555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:26.232721 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 449/912] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:26.500479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b053df-a100-4cea-b436-98e64a0d1fa3 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:26.603117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b053df-a100-4cea-b436-98e64a0d1fa3 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:26.639318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:26.640104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.640454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.640779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.652637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:52:26.657462 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:26.658623 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:26.659866 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 464/913] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:52:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:26.664804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:26.705470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:26.762868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d111-010b-444c-8e28-57d623b7ca48 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:26.796073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:26.796683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.796962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.797239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.797904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:26.798209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:26.799576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:26.807807 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 450/914] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:52:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:26.814266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:26.866967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:26.915421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d3da-8586-4edc-967d-3a401308ce58 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:26.947770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:26.948395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.948672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.948941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.959696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:26.961150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:26.976413 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:26.977516 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:26.978559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:26.978559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:26.978559 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:26.978559 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 465/915] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:26.983997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.084740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:27.087805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b799ec-4221-4acb-8077-99162de9436d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.122396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.123086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.123372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.123681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.124384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:27.124721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.126288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.137512 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 451/916] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:27.143464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.260812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059011-7063-4eb6-9eeb-96e602c6960e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.262470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:27.297122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.297774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.298061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.298322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.307013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.308548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:52:27.320850 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:27.321968 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:27.323339 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 466/917] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:27.328152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.434916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04290b-3569-4f84-8a4f-669d47102228 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.469264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.469877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.470162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.470441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.471370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:27.471711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.473151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.495333 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 452/918] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:27.501393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.602945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07dcb5-5d12-4942-b583-d7780b2c5b62 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.636835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.637502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.637774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.638047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.649354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.650812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:27.665111 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:27.666128 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:27.667057 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:27.667057 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:27.667057 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:27.667057 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 467/919] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:27.674281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.694792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:27.775436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cbd24-40c8-4716-a87b-3b51882353c0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.809238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.809851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.810119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.810391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.811031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:27.811364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.812776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.824800 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 453/920] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:27.830560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:27.847335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:27.932583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc2b2-3f86-459b-a640-374b05a9f74b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:27.968615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:27.969171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:27.969450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:27.969720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:27.976987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:27.978545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:52:27.990635 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:27.991715 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:27.992707 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 468/921] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:27.997992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:28.075526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:28.096869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae497e-26e6-4ff4-b14a-950dc6fc8eb1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:28.132526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:28.133159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:28.133414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:28.133690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:28.134425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:28.134736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:28.136196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc655f-819e-41e9-934f-13ebe6cfa680 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:28.148404 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 454/922] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:28.154480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:28.229315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:28.254439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1390-76e4-40c9-8eee-f6d97618688c None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:28.287985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:28.288637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:28.288922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:28.289200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/0133918a941b4c3ba07842d5b4deabbb/roles/1f88c6bf07704438bb2935318bd3fd5d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:28.302999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=0133918a941b4c3ba07842d5b4deabbb, role_id=1f88c6bf07704438bb2935318bd3fd5d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:28.304463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c65e61-fdf7-40af-b7d1-0ea6a595096f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:28.382587 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 469/923] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:52:28 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/0133918a941b4c3ba07842d5b4deabbb/roles/1f88c6bf07704438bb2935318bd3fd5d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:52:30.565135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194436-2acb-4338-9b6a-897717c4642c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:30.565719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194436-2acb-4338-9b6a-897717c4642c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:30.565719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194436-2acb-4338-9b6a-897717c4642c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:30.567647 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 455/924] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:30.573037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:30.573458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:30.573597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:30.591522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:30.592754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:30.659719 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 470/925] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:30.929102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64194436-2acb-4338-9b6a-897717c4642c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.029483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64194436-2acb-4338-9b6a-897717c4642c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.065744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.066381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.066663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.066935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.074089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:31.077839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:52:31.083230 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:31.084359 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:31.085261 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 456/926] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Oct 30 00:52:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:31.090457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.145130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:31.188065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb7c3d-32f7-4a9d-98b3-742ea2e17949 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.222304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.222912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.223194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.223470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.224165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:31.224470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:31.225841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:31.233973 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 471/927] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:31.240808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.339343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c52f-574d-474e-8c27-57f0859cef54 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.377168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.377828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.378108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.378382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.389107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:31.390585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:31.405002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:31.409951 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:31.411533 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:31.411533 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:31.411533 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:31.411533 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 457/928] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:31.412599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.514486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121545c6-e670-4cdc-a06f-689382a4cd85 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.530424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:31.548798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.552583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.552583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.552583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.552583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:31.552583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:31.552932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:31.564811 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 472/929] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:31.570332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.619364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:31.666848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a6c6f-ed47-4002-85f7-dd2b5d7c5137 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.700362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.701033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.701313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.701585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.709090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:31.710512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:52:31.721319 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:31.722445 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:31.724231 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 458/930] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:31.728811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.828141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795df71-9021-4c57-9158-711f084ce263 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:31.862735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:31.863343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:31.863642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:31.863914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:31.864639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:31.864940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:31.866360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:31.883183 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 473/931] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:31.889178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:31.990911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a94a-03cc-4764-aa68-accc135ed40d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.001101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:32.024058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.024667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.024949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.025225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.091252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.092739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.190083 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 459/932] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:52:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:52:32.195567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:32.232282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:32.301465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcf7d1-8ef7-4209-8dd0-31ad1fad1d66 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.335682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.336314 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.336591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.336865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.350635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.350635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:32.364109 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:32.365329 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:32.366300 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:32.366300 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:32.366300 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:32.366300 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 474/933] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:32.371413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:32.383966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:32.475093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4e16e-3046-4777-824b-600b64f5a9a0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.510227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.510837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.511114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.511386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.512062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:32.512368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.513715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.524772 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 460/934] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:32.530623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:32.616727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:32.631559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edd5b-f639-48ad-a19d-c38b3d1b0bb1 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.665145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.665894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.666047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.666317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.673186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.680258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:52:32.685844 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:32.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:32.688045 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 475/935] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:32.692808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:32.762791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:32.793522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a569e6-a251-48df-b189-95c45b39bf5c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.825885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.826487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.827248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.827523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.828246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:32.828550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.830007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af935a-1caf-41d6-a698-6d2982a05930 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.842880 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 461/936] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:32.849652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:32.952008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206635a0-995e-465c-a923-8adbd016a557 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:32.987342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:32.987987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:32.988268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:32.988540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:32.989267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] RBAC: Adding query filter params (role.id=1f88c6bf07704438bb2935318bd3fd5d, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=0133918a941b4c3ba07842d5b4deabbb) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:32.989576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:32.991022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488267a-a6a9-4c30-8c3e-2b04f586525c None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:32.996827 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 476/937] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:52:32 2022] GET /identity/v3/role_assignments?user.id=0133918a941b4c3ba07842d5b4deabbb&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=1f88c6bf07704438bb2935318bd3fd5d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:35.195887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00923ba8-14fd-4439-926c-479b4a2dcb92 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:35.196477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00923ba8-14fd-4439-926c-479b4a2dcb92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:35.196477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00923ba8-14fd-4439-926c-479b4a2dcb92 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:35.197828 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 462/938] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:35.203145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:35.203598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:35.203713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:35.222655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:35.223898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:35.226715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:35.291923 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 477/939] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:35.557068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00923ba8-14fd-4439-926c-479b4a2dcb92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:35.664519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00923ba8-14fd-4439-926c-479b4a2dcb92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:35.694193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:35.700159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:35.700789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:35.701073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:35.701349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:35.711559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:52:35.716408 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:35.717527 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:52:35.718525 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:35.719841 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 463/940] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:35.723790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:35.825256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82038-20c3-41f2-88c1-f3fc8d178f52 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:35.860366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:35.861019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:35.861304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:35.861581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:35.862299 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:35.862610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:35.864565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:35.873055 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 478/941] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:35.879463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:35.981386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b59be6-b2af-40ba-8f13-00cc18a710e7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.016254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.016891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.017174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.017446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.027468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.029016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:36.044644 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:36.045868 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:36.047127 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:36.047127 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:36.047127 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:36.047127 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 464/942] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:52:36.052313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.055268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:36.155800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3376-b9fe-4740-aca6-ce202ea6ff07 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.194805 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.195429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.195778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.196099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.196789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:36.197093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.198476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.210668 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 479/943] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:36.216479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.263482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:36.321272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e962a8-4c46-42c5-9efc-28da16954dfe None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.355510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.356176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.356460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.356751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.364509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.365964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:52:36.377185 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:36.378691 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:36.379965 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 465/944] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:36.384260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.447382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:36.488723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c344a-90e0-40ed-a353-6bdb4efa1296 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.526012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.526638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.526922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.527199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.527985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:36.528295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.529771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.550392 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 480/945] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:36.556490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.663734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ab06d-11ac-4eb1-a430-bb247acd9fc1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.665437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:36.700133 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.700686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.700965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.701237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.712149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.713640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:36.728555 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:36.730002 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:36.731072 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:36.731072 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:36.731072 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:36.731072 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 466/946] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:36.735798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.834522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f02a40-0e3c-4e7b-b71a-8e07e50c3bb9 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:36.854873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:36.868461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:36.869071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:36.869350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:36.869626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:36.870263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:36.870566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:36.871989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:36.883843 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 481/947] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:36.889649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:36.989090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e533c57-2bc7-492b-9828-e97e29565a70 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:37.021631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:37.022252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:37.022533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:37.022809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:37.029693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:37.031169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:52:37.043009 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:37.048087 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:37.049286 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 467/948] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:37.050094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:37.149293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a3aa-052b-435f-b168-fadc7dd9f89e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:37.182865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:37.183474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:37.183780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:37.184059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:37.184747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:37.185053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:37.186475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb612-faf4-4248-af79-e301e4e32368 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:37.199018 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 482/949] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:37.205093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:37.221181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:37.303029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98568fa-517a-4ca8-8533-ed487b2ba348 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:37.337251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:37.337965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:37.338250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:37.338528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] PATH_INFO: `/v3/projects/97a13536393943d4b1e22deb299678b1/users/0133918a941b4c3ba07842d5b4deabbb/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:37.340653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a13536393943d4b1e22deb299678b1, user_id=0133918a941b4c3ba07842d5b4deabbb, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:37.342107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208744c-b42e-4b5b-814a-ec89a0f94f92 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:37.437737 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 468/950] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Oct 30 00:52:37 2022] PUT /identity/v3/projects/97a13536393943d4b1e22deb299678b1/users/0133918a941b4c3ba07842d5b4deabbb/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:52:39.649035 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550a46-c64a-4d91-b07a-62803f92686f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:39.649635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550a46-c64a-4d91-b07a-62803f92686f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:39.649728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550a46-c64a-4d91-b07a-62803f92686f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:39.651180 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 483/951] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:39.656935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:39.661167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:39.661167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:39.678340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:39.679536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:39.747144 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 469/952] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:40.019726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e550a46-c64a-4d91-b07a-62803f92686f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.065388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:40.129193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e550a46-c64a-4d91-b07a-62803f92686f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.165772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:40.166437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:40.166729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:40.167001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:40.179058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:52:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:40.185277 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:40.186328 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 484/953] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:40.192181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.195330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:40.293386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b65b09-4014-4c5a-89d1-3996a60dccf4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.327736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:40.328352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:40.328634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:40.329810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:40.329810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:40.330135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:40.332229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:40.342427 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 470/954] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:40.348384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.449777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:40.453158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb675c-18c0-46d7-a07f-df1101003a24 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.489937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:40.490601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:40.490880 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:40.491148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:40.502015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:40.503504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:40.518432 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:40.519680 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:40.520764 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:40.520764 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:40.520764 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:40.520764 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 485/955] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:40.525551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.595131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:40.631354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc0e69-23c4-4b0c-8ba4-f0dd1030c202 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.665605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:40.666246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:40.666820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:40.666820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:40.667421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:40.667771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:40.669135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:40.684206 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 471/956] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:40.690184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.803680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a4e-3e1f-4100-99cf-ee383e29d48b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.837254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:40.837884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:40.838162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:40.838432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:40.846099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:40.847497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:52:40.859025 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:40.869473 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:40.870499 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 486/957] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:40.871123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:40.966364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c792cd0-2f0c-476f-bbb1-8ac720ff7246 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:40.992409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:41.000946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.001605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.001881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.002159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.002881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:41.003804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.005220 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.023022 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 472/958] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Oct 30 00:52:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:41.028961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:41.044356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:41.132878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15dc4a-b324-4ede-8554-88c4c21af12b None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:41.167667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.168286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.168567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.168837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.237156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.239165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.252527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:41.339872 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 487/959] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:52:41.345628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:41.445214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf06ab8-87eb-4938-812f-5ee804667c66 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:41.478415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.479301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.479575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.479867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.490396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.491920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:52:41.506684 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:41.515423 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:52:41.516578 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:52:41.516578 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:52:41.516578 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:41.516578 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 473/960] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:52:41.516807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:41.616486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd904d1-10be-4e30-a427-fb8b48f98d35 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:41.630561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:41.654451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.655211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.655554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.655868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.656521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:41.656828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.658284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.673950 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 488/961] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:41.676516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:41.712070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:41.775332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c6a6e-1e6d-41f2-8b96-a934ac1b01c4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:41.808253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.808880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.809161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.809433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.816535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.818273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:52:41.829728 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:41.830860 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: ERROR keystone.server.flask.application Oct 30 00:52:41.841161 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 474/962] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:41.841796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:41.938340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316e6e1-594f-4c39-aa79-8ff0f774616f None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:41.972621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:41.973258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:41.973484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:41.973777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:41.974479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:41.974793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:41.976247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6c3e3-82b0-4b55-9c38-6ca4e92942b8 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:41.989406 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 489/963] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:41.995696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:42.093071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:42.099183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8464cd-0dcd-4103-93f5-7e3ce99a2c17 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:42.133578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:42.134187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:42.134468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:42.134746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:42.135481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] RBAC: Adding query filter params (role.id=59156af11c8d4db5aa049e4462ed2ed0, scope.project.id=97a13536393943d4b1e22deb299678b1, user.id=0133918a941b4c3ba07842d5b4deabbb) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:42.135861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:42.137270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444f409-9998-40e8-86f6-b19acec19c8e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:42.143001 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 475/964] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Oct 30 00:52:41 2022] GET /identity/v3/role_assignments?user.id=0133918a941b4c3ba07842d5b4deabbb&scope.project.id=97a13536393943d4b1e22deb299678b1&role.id=59156af11c8d4db5aa049e4462ed2ed0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:44.331757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9730d3-2c2f-4e0d-82a6-9fd6016566c0 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:44.332321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9730d3-2c2f-4e0d-82a6-9fd6016566c0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:44.332321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9730d3-2c2f-4e0d-82a6-9fd6016566c0 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:44.333685 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 490/965] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:44.339164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:44.339686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:44.339785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:44.358144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:44.359348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:44.427266 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 476/966] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:44.699221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9730d3-2c2f-4e0d-82a6-9fd6016566c0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:44.730843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:44.803743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9730d3-2c2f-4e0d-82a6-9fd6016566c0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:44.837802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:44.838436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:44.838719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:44.838987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:44.839558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:44.841044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:44.850724 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:44.852108 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:44.852963 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 491/967] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:52:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:44.858250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:44.949202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:44.958705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3baf9-fbfa-4fcc-9ba0-7dc84c98dc3f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:44.992712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:44.993317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:44.993599 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:44.993868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:44.994493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:44.994799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:44.996195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993bcd3-362d-401e-a8c7-e77b5d744af2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:45.001285 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 477/968] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Oct 30 00:52:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:45.007191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:45.103978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:45.111083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b1a2-6c71-43e2-bb50-08e34f2970fc None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:45.146030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:45.146660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:45.146926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:45.147201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] PATH_INFO: `/v3/services` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:45.147861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:45.148167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:45.149541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b9b5-7797-4d9a-abeb-135ceba57f64 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:45.153899 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 492/969] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Oct 30 00:52:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:47.831718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92251a-7595-4f1c-84f8-5ae16847cc0d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:47.832286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92251a-7595-4f1c-84f8-5ae16847cc0d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:47.832728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92251a-7595-4f1c-84f8-5ae16847cc0d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:47.835278 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 478/970] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:47.841136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69b8a0-b633-4844-b993-9011f7fa2c28 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:47.841600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69b8a0-b633-4844-b993-9011f7fa2c28 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:47.841705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69b8a0-b633-4844-b993-9011f7fa2c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:47.860470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-fc69b8a0-b633-4844-b993-9011f7fa2c28 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:47.862170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fc69b8a0-b633-4844-b993-9011f7fa2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:47.932576 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 493/971] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:52:48.200159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92251a-7595-4f1c-84f8-5ae16847cc0d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:48.302574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92251a-7595-4f1c-84f8-5ae16847cc0d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:48.337206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:48.337940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:48.338220 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:48.338492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:48.339159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:48.340531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984d93-63b3-48b7-9a56-134b2962bd04 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:48.414797 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 479/972] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Oct 30 00:52:48 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:52:50.626480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec8c96-f5c2-46ab-b84d-712e42a71c47 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:50.627095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec8c96-f5c2-46ab-b84d-712e42a71c47 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:50.627095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec8c96-f5c2-46ab-b84d-712e42a71c47 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:50.628484 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 494/973] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:50.634258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:50.634659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:50.634864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:50.653554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:50.654746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:50.665300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:50.728006 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 480/974] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:50.996204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec8c96-f5c2-46ab-b84d-712e42a71c47 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:51.089722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:51.100610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec8c96-f5c2-46ab-b84d-712e42a71c47 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:51.136179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:51.136827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:51.137249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:51.137515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:51.138113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:51.139502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:51.149890 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:51.151263 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:51.152404 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 495/975] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:52:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:52:51.157568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:51.261328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e3687-9f1b-4bee-8b82-409c73f12ffa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:51.295219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:51.295858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:51.296140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:51.296410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:51.297035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:51.297333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:51.298699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:51.303745 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 481/976] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Oct 30 00:52:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:51.309784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:51.413429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359d033-7c13-452d-910b-5c33427e071e None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:51.448697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:51.449308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:51.449591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:51.449871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:51.450580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] RBAC: Adding query filter params (interface=public, service_id=a33c204a76964ee0b12cc91827a40f0c, region_id=RegionOne) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:51.450893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:51.452485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bc46-ee18-470f-b23b-e7b27bae7308 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:51.458385 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 496/977] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Oct 30 00:52:51 2022] GET /identity/v3/endpoints?service_id=a33c204a76964ee0b12cc91827a40f0c&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:51.464313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:51.468445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:51.568738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa8de-b258-42a7-8ec6-e570d4adf50d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:51.603103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:51.603732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:51.604013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:51.604289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:51.604889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:51.606333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc2d7a-43c1-426d-af8f-ef29e4b30c84 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:51.613639 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 482/978] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Oct 30 00:52:51 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:52:53.793829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3a81-9d47-4c99-8211-83c58cedd914 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:53.794380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3a81-9d47-4c99-8211-83c58cedd914 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:53.794380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3a81-9d47-4c99-8211-83c58cedd914 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:53.795707 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 497/979] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 00:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:52:53.801495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:53.807985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:53.807985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:53.820559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:53.821739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:53.884868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:53.890499 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 483/980] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 00:52:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:52:54.160467 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3a81-9d47-4c99-8211-83c58cedd914 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:54.179728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:54.261851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3a81-9d47-4c99-8211-83c58cedd914 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:54.296504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:54.297147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:54.297422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:54.297693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:54.298282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:54.300372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:52:54.309524 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:52:54.310717 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application with Lock( Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: ERROR keystone.server.flask.application Oct 30 00:52:54.311928 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 498/981] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Oct 30 00:52:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:52:54.317569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:54.413900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb20db-9b50-43e8-b764-96be3f34741c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:54.450545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:54.451296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:54.451578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:54.451879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:54.452514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:54.452819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:54.454211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264e2a6-2d29-4cd6-8ef8-2969d2a519d1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:54.459345 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 484/982] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Oct 30 00:52:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:54.465387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:54.546913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:54.566009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896bfd-6e93-4e71-8ea3-75ed8cadd156 None admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:54.599366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:54.600009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:54.600287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:54.600602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:54.601253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:54.602893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799ae87-79dd-4286-a6cd-853e92141384 None admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:54.762276 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 499/983] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 00:52:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:53:04.456322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=69 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:04.757646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:31.810505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd6b1-e0f5-4c74-8e77-17949293267f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:31.811009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd6b1-e0f5-4c74-8e77-17949293267f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:31.811228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd6b1-e0f5-4c74-8e77-17949293267f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:31.812791 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 485/984] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:31.817640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b495ab6-f5e0-4f5c-8c3b-4fc3bae9b88c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:31.818068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b495ab6-f5e0-4f5c-8c3b-4fc3bae9b88c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:31.818339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b495ab6-f5e0-4f5c-8c3b-4fc3bae9b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:31.960971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0b495ab6-f5e0-4f5c-8c3b-4fc3bae9b88c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:31.962104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0b495ab6-f5e0-4f5c-8c3b-4fc3bae9b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:32.128641 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 500/985] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:32.430592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:32.431176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:32.431668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:32.433485 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 486/986] 10.222.0.26 () {54 vars in 965 bytes} [Sun Oct 30 01:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:32.439646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b879-08d7-42a6-b816-1a6a9aaad248 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:32.439941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b879-08d7-42a6-b816-1a6a9aaad248 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:32.440221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b879-08d7-42a6-b816-1a6a9aaad248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:32.525410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b940b879-08d7-42a6-b816-1a6a9aaad248 None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:32.526568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b940b879-08d7-42a6-b816-1a6a9aaad248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:32.548848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:32.609772 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 501/987] 10.222.0.26 () {58 vars in 1062 bytes} [Sun Oct 30 01:00:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:32.619164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:32.621086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:32.757847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9af98-f750-44f3-84c9-b98c1337dfce None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:32.852350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:32.853054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:32.853522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:32.853885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:32.855820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:32.924180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:32.989461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:32.990929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562174d7-22a1-46d0-bb18-791979d185c7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:33.012214 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 487/988] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:35.561603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:35.562122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:35.562122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:35.563413 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 502/989] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:35.570248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14af2-1f48-4a46-b8be-d121def6e6a8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:35.570744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14af2-1f48-4a46-b8be-d121def6e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:35.571182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14af2-1f48-4a46-b8be-d121def6e6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:35.653465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-63a14af2-1f48-4a46-b8be-d121def6e6a8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:35.654751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-63a14af2-1f48-4a46-b8be-d121def6e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:35.751112 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 488/990] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:36.032442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:36.034015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:36.138773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155e17-5c0a-4ed7-8b10-86bc3a30f619 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:36.210164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:36.212475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:36.213087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:36.213367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:36.213640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:36.215236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:36.323518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:36.324897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f855-f213-453b-b0d5-218e41360cae service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:36.343179 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 503/991] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:39.295480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:39.296082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:39.296082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:39.297448 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 489/992] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:39.303200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5f71-0809-459e-8d4f-0b3847a16406 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:39.303649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5f71-0809-459e-8d4f-0b3847a16406 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:39.303874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5f71-0809-459e-8d4f-0b3847a16406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:39.372743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e45f5f71-0809-459e-8d4f-0b3847a16406 None None] MFA Rules not processed for user `6b908eebe0424e2fb3572a3d75adefe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:39.373968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e45f5f71-0809-459e-8d4f-0b3847a16406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:39.432885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:39.482529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 504/993] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:39.768808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:39.770366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:39.780114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:39.872335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f1c62-0ca4-4dab-b1c8-6d15da8150a9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:39.945955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:39.946475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:39.946755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:39.947028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:39.948633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:40.010029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:40.056964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:40.058397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f57518-2741-4025-96df-ddba2a1d1fea service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:40.077930 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 490/994] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:43.143561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:43.144118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:43.144247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:43.145515 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 505/995] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:43.150810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3072-c8dd-4513-9c54-7e506aac4286 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:43.151234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3072-c8dd-4513-9c54-7e506aac4286 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:43.151364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3072-c8dd-4513-9c54-7e506aac4286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:43.217011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7cca3072-c8dd-4513-9c54-7e506aac4286 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:43.218232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7cca3072-c8dd-4513-9c54-7e506aac4286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:43.313781 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 491/996] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:43.600028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:43.601561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:43.705757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2670a5-5e08-4a22-8567-3f0362f27268 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:43.779082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:43.779725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:43.780008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:43.780287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:43.781902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:43.790225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:43.892869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:43.894242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6dbc8-fc14-48c9-8c6a-f6601dfd6091 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:43.912531 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 506/997] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:47.296581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:47.297124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:47.297124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:47.298463 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 492/998] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:47.304427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1072d4-82c5-4d63-bb5a-c487db0f530e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:47.304861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1072d4-82c5-4d63-bb5a-c487db0f530e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:47.305115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1072d4-82c5-4d63-bb5a-c487db0f530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:47.372314 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7e1072d4-82c5-4d63-bb5a-c487db0f530e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:47.373554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7e1072d4-82c5-4d63-bb5a-c487db0f530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:47.438544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:47.482350 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 507/999] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:47.766971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:47.769251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:47.789483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:47.877428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f48fa-f99b-491b-b20e-01e4f922403a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:47.952149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:47.953022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:47.953300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:47.953577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:47.955163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:48.026639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:48.068392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:48.069853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2578f0f-1e90-49b2-bee5-6b7d38950e57 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:48.088600 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 493/1000] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:50.317272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:50.317801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:50.317912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:50.319200 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 508/1001] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:00:50.324899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cec8-7e83-4abc-bfd2-c166475995ef None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:50.325310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cec8-7e83-4abc-bfd2-c166475995ef None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:50.325544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cec8-7e83-4abc-bfd2-c166475995ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:50.393663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2e00cec8-7e83-4abc-bfd2-c166475995ef None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:50.394910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2e00cec8-7e83-4abc-bfd2-c166475995ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:50.498829 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 494/1002] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:50.780530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:50.782138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:50.890917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6746d-19a0-4ba5-8ed5-077e229601da None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:50.964850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:50.965468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:50.965743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:50.966011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:50.967686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:50.985123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:51.078986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:51.080475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292c08d-919d-4d41-861d-dacac34e8007 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:51.098712 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 509/1003] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:53.344506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61e63-dda8-432b-aab9-94004d6b9721 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:53.344506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61e63-dda8-432b-aab9-94004d6b9721 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:53.345124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61e63-dda8-432b-aab9-94004d6b9721 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:53.346033 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 495/1004] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:53.351521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e2ab-2466-4917-a836-f5e03ebbd95e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:53.351957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e2ab-2466-4917-a836-f5e03ebbd95e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:53.352055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e2ab-2466-4917-a836-f5e03ebbd95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:53.422150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4de9e2ab-2466-4917-a836-f5e03ebbd95e None None] MFA Rules not processed for user `6b908eebe0424e2fb3572a3d75adefe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:53.423381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4de9e2ab-2466-4917-a836-f5e03ebbd95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:53.492889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:53.531116 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 510/1005] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:53.837215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:53.837509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:53.837831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:53.839256 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 496/1006] 10.222.0.26 () {54 vars in 965 bytes} [Sun Oct 30 01:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:53.845805 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183909-c150-43b8-b3b7-d931c1b8653e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:53.846131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183909-c150-43b8-b3b7-d931c1b8653e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:53.846413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183909-c150-43b8-b3b7-d931c1b8653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:53.932651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-86183909-c150-43b8-b3b7-d931c1b8653e None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:53.933889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-86183909-c150-43b8-b3b7-d931c1b8653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:54.014033 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 511/1007] 10.222.0.26 () {58 vars in 1062 bytes} [Sun Oct 30 01:00:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:54.023788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:54.025417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:54.056678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:54.134425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affd744-e39d-404a-9b91-c167c26dcfbd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:54.213387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:54.213997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:54.214281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:54.214563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:54.216279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:54.296983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:54.328673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:54.330105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d13c8-069d-4cbc-9891-8d8b5139b2ec service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:54.349754 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 497/1008] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:57.484630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:57.485161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:57.485161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:57.486521 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 512/1009] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:00:57.492125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1359db-eff3-4274-b3f1-db9f390ad040 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:57.492586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1359db-eff3-4274-b3f1-db9f390ad040 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:57.492680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1359db-eff3-4274-b3f1-db9f390ad040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:57.559910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3e1359db-eff3-4274-b3f1-db9f390ad040 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:57.561033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3e1359db-eff3-4274-b3f1-db9f390ad040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:57.660255 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 498/1010] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:57.937823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:57.939403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:58.009538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:58.045223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8231c-e313-48b7-a749-8c8aa3bebb9f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:58.124693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:58.125314 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:58.125591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:58.125860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:58.128095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:58.238470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:58.239849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccb3d6-04e3-4099-878e-db2fc9630e9a service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:58.249753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:58.258522 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 513/1011] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:00.487214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:00.487214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:00.487214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:00.487975 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 499/1012] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:01:00.493091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da15e-abcf-4fb4-bcb8-dff5f08e9be8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:00.493547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da15e-abcf-4fb4-bcb8-dff5f08e9be8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:00.493714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da15e-abcf-4fb4-bcb8-dff5f08e9be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:00.559799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-df1da15e-abcf-4fb4-bcb8-dff5f08e9be8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:00.561050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-df1da15e-abcf-4fb4-bcb8-dff5f08e9be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:00.659263 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 514/1013] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:00.945790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:00.947492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:00.988071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:01.050942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78309eda-0eb2-4bd6-8b18-7c00d4fdbd03 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:01.123038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:01.123652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:01.123933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:01.124201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:01.125776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:01.206810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:01.232767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:01.234222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb6f10-3bcf-4d51-a4e0-a29375369cbc service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:01.252868 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 500/1014] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:04.070447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:04.070969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:04.070969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:04.072258 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 515/1015] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:01:04.077349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d231-af77-43b6-94e0-718d44a8ad1f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:04.077772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d231-af77-43b6-94e0-718d44a8ad1f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:04.077907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d231-af77-43b6-94e0-718d44a8ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:04.144680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3364d231-af77-43b6-94e0-718d44a8ad1f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:04.145903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3364d231-af77-43b6-94e0-718d44a8ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:04.242442 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 501/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:04.525722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:04.527338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:04.609230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:04.636378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91191a3-fa67-48fa-961e-abc7c67dd05f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:04.715800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:04.716418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:04.716700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:04.716972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:04.718569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:04.829568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:04.831091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc41af-d333-4806-92ef-94e8a7806d29 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:04.849437 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 516/1017] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:09.320040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:09.320604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:09.320604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:09.321979 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 502/1018] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:09.327519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d248-8e9f-4aa8-b4f5-bf28faeb8575 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:09.327985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d248-8e9f-4aa8-b4f5-bf28faeb8575 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:09.328637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d248-8e9f-4aa8-b4f5-bf28faeb8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:09.334365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:09.397319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9872d248-8e9f-4aa8-b4f5-bf28faeb8575 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:09.398530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9872d248-8e9f-4aa8-b4f5-bf28faeb8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:09.501297 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 517/1019] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:09.785944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:09.787481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:09.838917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:09.900351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae0aa8-7a9b-4863-9524-2f18d7e64e0e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:09.972501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:09.973106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:09.973386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:09.973663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:09.975250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:10.070369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:10.087003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:10.088479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56588d0-623a-40d4-8a4d-4913a39fdc08 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:10.106928 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 503/1020] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:13.516446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:13.516971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:13.516971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:13.518867 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 518/1021] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:13.524288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb495a-6822-453d-b85f-babfb22a7ce9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:13.524690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb495a-6822-453d-b85f-babfb22a7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:13.524799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb495a-6822-453d-b85f-babfb22a7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:13.594801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-48eb495a-6822-453d-b85f-babfb22a7ce9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:13.595929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-48eb495a-6822-453d-b85f-babfb22a7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:13.705916 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 504/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:13.996258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:13.997854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:14.116818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:14.134708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0c02-2e64-4087-9612-34d842c5c8cf None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:14.217690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:14.219771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:14.219771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:14.219771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:14.220628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:14.353719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:14.355250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b41074-37be-4606-a690-9ad79e2aa010 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:14.378194 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 519/1023] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:18.810023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:18.810606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:18.810837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:18.812299 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 505/1024] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:01:18.819273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af458a-e542-4d79-9107-9524f8712134 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:18.821358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af458a-e542-4d79-9107-9524f8712134 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:18.821358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af458a-e542-4d79-9107-9524f8712134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:18.837313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:18.899624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a1af458a-e542-4d79-9107-9524f8712134 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:18.901620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a1af458a-e542-4d79-9107-9524f8712134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:19.012318 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 520/1025] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:19.296095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:19.297654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:19.359658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:19.408088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5af54a-67fe-4d1d-bf8f-916695878819 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:19.489435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:19.490055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:19.490336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:19.490611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:19.492364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:19.601404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:19.605063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:19.606578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15aa2f-3e20-4464-b943-890ab483bda3 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:19.626202 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 506/1026] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:24.729230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5d3e-9d43-4cb7-95a5-65eb79d66ddf None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:24.729732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5d3e-9d43-4cb7-95a5-65eb79d66ddf None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:24.729827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad5d3e-9d43-4cb7-95a5-65eb79d66ddf None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:24.731157 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 521/1027] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:24.739015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68669c2-32eb-45cf-b2b4-fe73c2659434 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:24.739015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68669c2-32eb-45cf-b2b4-fe73c2659434 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:24.739015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68669c2-32eb-45cf-b2b4-fe73c2659434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:24.846071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d68669c2-32eb-45cf-b2b4-fe73c2659434 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:24.847558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d68669c2-32eb-45cf-b2b4-fe73c2659434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:25.026626 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 507/1028] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:01:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:29.010547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=59 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:29.621908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fce-6f04-4907-b7c9-6354b40fe526 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:29.622179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fce-6f04-4907-b7c9-6354b40fe526 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:29.622788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fce-6f04-4907-b7c9-6354b40fe526 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:29.624272 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 522/1029] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:29.630844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b00fcd-a6d6-4a91-8f69-f2f730583cc2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:29.631416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b00fcd-a6d6-4a91-8f69-f2f730583cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:29.631416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b00fcd-a6d6-4a91-8f69-f2f730583cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:29.717174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:29.745072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-14b00fcd-a6d6-4a91-8f69-f2f730583cc2 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:29.746454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-14b00fcd-a6d6-4a91-8f69-f2f730583cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:29.902240 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 508/1030] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:31.002875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ce67-4aab-4f80-8111-7ffa479413c3 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:31.004133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ce67-4aab-4f80-8111-7ffa479413c3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:31.004133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ce67-4aab-4f80-8111-7ffa479413c3 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:31.005957 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 523/1031] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:01:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:31.014343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de48c0-436e-4fbb-b619-3f6e3dd386bc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:31.015199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de48c0-436e-4fbb-b619-3f6e3dd386bc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:31.015199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de48c0-436e-4fbb-b619-3f6e3dd386bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:31.103553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c6de48c0-436e-4fbb-b619-3f6e3dd386bc None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:31.104877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6de48c0-436e-4fbb-b619-3f6e3dd386bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:31.152140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:31.251081 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 509/1032] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:01:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:01:34.858398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8be6c-924f-4316-95a3-a98d1a9a4ad5 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:34.859266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8be6c-924f-4316-95a3-a98d1a9a4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:34.859546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8be6c-924f-4316-95a3-a98d1a9a4ad5 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:34.861171 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 524/1033] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:34.867156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990ac8-6ccf-40a3-b46f-41e7a36e0f77 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:34.867694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990ac8-6ccf-40a3-b46f-41e7a36e0f77 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:34.868040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990ac8-6ccf-40a3-b46f-41e7a36e0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:34.954085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a7990ac8-6ccf-40a3-b46f-41e7a36e0f77 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:34.955666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a7990ac8-6ccf-40a3-b46f-41e7a36e0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:35.066413 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 510/1034] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:35.368057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:35.368742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:35.369160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:35.370940 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 525/1035] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:01:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:35.379516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a933a76-ced9-4359-89d8-3fc5a8c66ddf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:35.379834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a933a76-ced9-4359-89d8-3fc5a8c66ddf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:35.380196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a933a76-ced9-4359-89d8-3fc5a8c66ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:35.396662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:35.466808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6a933a76-ced9-4359-89d8-3fc5a8c66ddf None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:35.468055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a933a76-ced9-4359-89d8-3fc5a8c66ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:35.611897 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 511/1036] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:35.613208 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:35.615394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:35.794164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b13c-b039-4c48-8923-ed2aaf20dfa3 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:35.873006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:35.914829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:35.915445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:35.915756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:35.916034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:35.917651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:36.021055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:36.022881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2cbcc-0b96-44fc-8a1e-1934bca1b5bc service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:36.044340 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 526/1037] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:38.465597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:38.466103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:38.466103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:38.467396 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 512/1038] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:38.474478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fafca8-2bd4-4b02-b1ac-7141104e9d22 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:38.474478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fafca8-2bd4-4b02-b1ac-7141104e9d22 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:38.475068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fafca8-2bd4-4b02-b1ac-7141104e9d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:38.578783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-93fafca8-2bd4-4b02-b1ac-7141104e9d22 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:38.580825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-93fafca8-2bd4-4b02-b1ac-7141104e9d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:38.627255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:38.705141 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 527/1039] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:39.002158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:39.003981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:39.078571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:39.189586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd58e3-df07-4c7c-954b-fb5f54f1fbe3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:39.321786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:39.322429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:39.322709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:39.322981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:39.325429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:39.354069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:39.427834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:39.430297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84bb97-372f-4c79-8e85-747adcd5aea3 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:39.459353 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 513/1040] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:42.077535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c68e-cce5-4e89-82b0-911b3ba0ebea None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:42.078141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c68e-cce5-4e89-82b0-911b3ba0ebea None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:42.078141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c68e-cce5-4e89-82b0-911b3ba0ebea None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:42.079880 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 528/1041] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:42.085424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726b8b-156d-4cdd-838e-8191f16f4af4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:42.085938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726b8b-156d-4cdd-838e-8191f16f4af4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:42.086347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726b8b-156d-4cdd-838e-8191f16f4af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:42.161997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e2726b8b-156d-4cdd-838e-8191f16f4af4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:42.163367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e2726b8b-156d-4cdd-838e-8191f16f4af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:42.271054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:42.273769 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 514/1042] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:42.572713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:42.573194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:42.573679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:42.575276 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 529/1043] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:42.581434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a12b0-cd1b-47f6-893e-9baa6156374c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:42.581725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a12b0-cd1b-47f6-893e-9baa6156374c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:42.582003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a12b0-cd1b-47f6-893e-9baa6156374c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:42.675249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e85a12b0-cd1b-47f6-893e-9baa6156374c None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:42.677054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e85a12b0-cd1b-47f6-893e-9baa6156374c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:42.806019 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 515/1044] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:01:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:01:42.813813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:42.816079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:42.978879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a13ee7-4230-42e3-9f58-0f03cac785be None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:43.000057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:43.106660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:43.107547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:43.107547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:43.107959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:43.109611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:43.208998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:43.209620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b02e1-0138-4fa6-9660-1ca44187b677 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:43.210142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b02e1-0138-4fa6-9660-1ca44187b677 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:43.210142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b02e1-0138-4fa6-9660-1ca44187b677 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:43.210618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83993108-cf18-4d22-9065-57a924e10fce service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:43.211680 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 516/1045] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:01:43.216430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16422f0-3a6e-4bf3-ab90-bc95ee1fffc1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:43.216714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16422f0-3a6e-4bf3-ab90-bc95ee1fffc1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:43.216989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16422f0-3a6e-4bf3-ab90-bc95ee1fffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:43.232367 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 530/1046] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:43.281822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:43.289891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b16422f0-3a6e-4bf3-ab90-bc95ee1fffc1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:43.291123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b16422f0-3a6e-4bf3-ab90-bc95ee1fffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:43.410595 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 517/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:01:43.750896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:43.750896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:43.750896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:43.752468 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 531/1048] 10.222.0.26 () {54 vars in 963 bytes} [Sun Oct 30 01:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:43.757460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b983-31b9-4b72-a16e-71a33bc362d0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:43.757748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b983-31b9-4b72-a16e-71a33bc362d0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:43.758027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b983-31b9-4b72-a16e-71a33bc362d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:43.847532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-63e3b983-31b9-4b72-a16e-71a33bc362d0 None None] MFA Rules not processed for user `5de8280cded549c58eeabd4e4dc37964`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:43.848825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-63e3b983-31b9-4b72-a16e-71a33bc362d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:43.968092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:43.979956 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 518/1049] 10.222.0.26 () {58 vars in 1060 bytes} [Sun Oct 30 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:43.991267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:43.999496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:44.044389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:44.166972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f14118-8167-449e-bae3-234fef4fa71d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:44.262123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:44.262872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:44.263292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:44.263708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:44.265792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:44.365574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:44.424467 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:44.426226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549aa77-cc83-49aa-abd1-9e2ffe6ee5df service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:44.451630 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 532/1050] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:45.683222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:45.683822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:45.683822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:45.685148 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 519/1051] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:45.691199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2009fe-a40c-452f-90e0-ae32d77b259d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:45.691719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2009fe-a40c-452f-90e0-ae32d77b259d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:45.691845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2009fe-a40c-452f-90e0-ae32d77b259d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:45.761130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ba2009fe-a40c-452f-90e0-ae32d77b259d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:45.762405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ba2009fe-a40c-452f-90e0-ae32d77b259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:45.872682 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 533/1052] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:46.166905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:46.169455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:46.349729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8bc0-61e9-43f0-b9cd-a9e1e23799f3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:46.429835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:46.486741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:46.487365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:46.487682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:46.487972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:46.489688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:46.584421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:46.585613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bfc8b-3089-4b0f-8539-70f75bb1b1d3 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:46.604821 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 520/1053] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:49.024134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:49.024683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:49.024683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:49.026134 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 534/1054] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:49.031455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddb28e-bc44-4d8b-8717-cc273ea55948 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:49.031946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddb28e-bc44-4d8b-8717-cc273ea55948 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:49.032667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddb28e-bc44-4d8b-8717-cc273ea55948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:49.130148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-66ddb28e-bc44-4d8b-8717-cc273ea55948 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:49.132138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-66ddb28e-bc44-4d8b-8717-cc273ea55948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:49.182037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:49.264300 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 521/1055] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:49.559222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:49.560780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:49.601477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:49.721254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15593a-09f5-49ff-b5ae-745b8236744a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:49.843130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:49.848914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:49.849528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:49.849808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:49.850108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:49.851865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:49.958473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:49.960391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14fccb-8752-4210-8c18-f613280b5394 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:49.980291 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 535/1056] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:52.365028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:52.365028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:52.365781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:52.368531 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 522/1057] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:52.372281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deea6-0236-4b3d-a5e9-c5590a2a7a49 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:52.372852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deea6-0236-4b3d-a5e9-c5590a2a7a49 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:52.372852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deea6-0236-4b3d-a5e9-c5590a2a7a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:52.452525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-fc9deea6-0236-4b3d-a5e9-c5590a2a7a49 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:52.453572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fc9deea6-0236-4b3d-a5e9-c5590a2a7a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:52.537774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:52.571659 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 536/1058] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:52.936290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:52.938057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:53.119961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9584b-28bc-41a7-902b-d757449d2dcc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:53.138043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:53.262075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:53.262597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:53.262878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:53.263151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:53.264796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:53.384927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:53.389369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d22d-6451-4c56-82ab-cf65fff8630a service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:53.412052 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 523/1059] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:55.823042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:55.823896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:55.823896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:55.824964 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 537/1060] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:01:55.830667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed2c-07f8-4156-aeae-420453cda13a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:55.831283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed2c-07f8-4156-aeae-420453cda13a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:55.831283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed2c-07f8-4156-aeae-420453cda13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:55.854911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:55.901777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6300ed2c-07f8-4156-aeae-420453cda13a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:55.903011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6300ed2c-07f8-4156-aeae-420453cda13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:56.007222 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 524/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:56.309915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:56.311441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:56.479981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012f5bd-a9c9-40bf-b2cf-5dfa31520e71 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:56.542386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:56.617382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:56.618271 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:56.618877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:56.619226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:56.621431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:56.734690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:56.736806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f1029-872b-43db-b574-2cb33256da8a service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:56.762843 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 538/1062] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:59.050171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:59.050692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:59.050692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:59.052080 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 525/1063] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:01:59.057690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d236a2-b406-4d01-9945-b77307baee61 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:59.058148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d236a2-b406-4d01-9945-b77307baee61 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:59.058230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d236a2-b406-4d01-9945-b77307baee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:59.136040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-31d236a2-b406-4d01-9945-b77307baee61 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:59.137275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-31d236a2-b406-4d01-9945-b77307baee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:59.146118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:59.246023 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 539/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:59.539361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:01:59.540928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:59.632976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:59.691648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b055-5060-4f58-9a9d-53753f472eae None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:01:59.807530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:01:59.808169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:59.808455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:59.808733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:59.810321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:59.890294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:59.920824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:01:59.923169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a8a20-dd56-4b5d-bd66-15090e976e34 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:01:59.942581 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 526/1065] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:00.321329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe4d1-9576-486a-80e9-6fe9cae84c89 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:00.321857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe4d1-9576-486a-80e9-6fe9cae84c89 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:00.321857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe4d1-9576-486a-80e9-6fe9cae84c89 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:00.323631 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 540/1066] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:00.329568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8a64-106d-4ca7-8290-76a7d919b92c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:00.330512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8a64-106d-4ca7-8290-76a7d919b92c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:00.330796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8a64-106d-4ca7-8290-76a7d919b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:00.405448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d7ba8a64-106d-4ca7-8290-76a7d919b92c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:00.406739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d7ba8a64-106d-4ca7-8290-76a7d919b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:00.527674 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 527/1067] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:00.843188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:00.843730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:00.844170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:00.846474 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 541/1068] 10.222.0.26 () {54 vars in 962 bytes} [Sun Oct 30 01:02:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:00.854998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3acd3-dcc0-48cc-97dc-dd2990746ab2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:00.855287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3acd3-dcc0-48cc-97dc-dd2990746ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:00.855560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3acd3-dcc0-48cc-97dc-dd2990746ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:00.911679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:00.949115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-35c3acd3-dcc0-48cc-97dc-dd2990746ab2 None None] MFA Rules not processed for user `aa1b9f055f6c4def891834a8114b04b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:00.950296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-35c3acd3-dcc0-48cc-97dc-dd2990746ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:01.037335 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 528/1069] 10.222.0.26 () {58 vars in 1059 bytes} [Sun Oct 30 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:01.045304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:01.046828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:01.157255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bc449-9a3f-4ca0-9a5e-47e15d54e0a0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:01.197737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:01.235565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:01.236121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:01.236409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:01.236680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:01.238312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:01.361043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:01.362533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9539dc-078d-4bf3-a0c4-45be70c4f466 service glance] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:01.383680 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 542/1070] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:02.317417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74529b7-ff73-44ef-b2e6-063a4af9d079 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:02.318281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74529b7-ff73-44ef-b2e6-063a4af9d079 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:02.318281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74529b7-ff73-44ef-b2e6-063a4af9d079 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:02.319313 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 529/1071] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:02.328853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28637f-6915-4331-abda-60835c94367f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:02.329353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28637f-6915-4331-abda-60835c94367f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:02.329658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28637f-6915-4331-abda-60835c94367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:02.357838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da293d71-6c97-4b1b-a686-f7cf3dde68b2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:02.358335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da293d71-6c97-4b1b-a686-f7cf3dde68b2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:02.358753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da293d71-6c97-4b1b-a686-f7cf3dde68b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:02.413882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3a28637f-6915-4331-abda-60835c94367f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:02.415126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3a28637f-6915-4331-abda-60835c94367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:02.435953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:02.441968 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-da293d71-6c97-4b1b-a686-f7cf3dde68b2 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:02.443245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-da293d71-6c97-4b1b-a686-f7cf3dde68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:02.540029 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 543/1072] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:02.548778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:02.556547 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 530/1073] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:02:02.615627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:02.616394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:02.616862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:02.619447 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 544/1074] 10.222.0.26 () {54 vars in 943 bytes} [Sun Oct 30 01:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:02.629923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:02.630307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:02.630593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:02.762378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] MFA Rules not processed for user `d6a6265c254040d9a0d8ae45b29d3073`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:02.763887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:02.893263 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 531/1075] 10.222.0.26 () {58 vars in 1040 bytes} [Sun Oct 30 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:02.902466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:02.903890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:02.904096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:02.915689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d010313-44b9-4c8a-bb9b-96fce459bc7b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:02.973157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:03.036836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:03.037374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:03.037660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:03.037945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:03.039648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:03.068601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a5e8-7a83-4101-bcac-3e6e8cec37d4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:03.074053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:03.142922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:03.145377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e5995-4ac9-447f-8da1-a803965feaae service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:03.162796 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 532/1076] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:02:03.228765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:03.229393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:03.229682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:03.229959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:03.231624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:03.331718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:03.333429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3dc78-255c-4012-9410-3feb9e8d0a89 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:03.352819 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 545/1077] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:05.877210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:05.878326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:05.878326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:05.879378 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 533/1078] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:02:05.885226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8090c-5c21-4487-be19-eeb1751eaa57 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:05.886037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8090c-5c21-4487-be19-eeb1751eaa57 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:05.886037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8090c-5c21-4487-be19-eeb1751eaa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:05.896275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:05.971891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-57e8090c-5c21-4487-be19-eeb1751eaa57 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:05.971891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57e8090c-5c21-4487-be19-eeb1751eaa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:06.119132 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 546/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:02:06.428070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:06.431291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:06.592098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:06.726558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c92e1-db02-474c-b903-ea142b6c31ae None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:06.949124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:06.951443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:06.952242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:06.952715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:06.956288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:07.121574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:07.153852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:07.159039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193d1bb-b2c5-4fde-8ead-a1acc5c888ba service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:07.188906 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 534/1080] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:10.525717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:10.526368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:10.526678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:10.529380 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 547/1081] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:02:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:10.538880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372056-81bc-4602-aa75-c812389ba3d9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:10.539448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372056-81bc-4602-aa75-c812389ba3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:10.539855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372056-81bc-4602-aa75-c812389ba3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:10.679739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-16372056-81bc-4602-aa75-c812389ba3d9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:10.681024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-16372056-81bc-4602-aa75-c812389ba3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:10.853417 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 535/1082] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:11.161254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:11.163337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:11.278601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:11.406765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf0078-31d4-49a4-a16c-cdb801057e1e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:11.564966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:11.565740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:11.565901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:11.566124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:11.567921 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:11.634389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:11.712578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:11.716564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1425ef-efee-4eda-bd08-a57fb242b20e service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:11.758685 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 548/1083] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:15.423841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:15.424473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:15.424473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:15.425752 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 536/1084] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:15.431263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ea8e-c0fa-48c1-9cd4-daa07e6e6e5e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:15.431758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ea8e-c0fa-48c1-9cd4-daa07e6e6e5e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:15.431849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ea8e-c0fa-48c1-9cd4-daa07e6e6e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:15.505850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bc20ea8e-c0fa-48c1-9cd4-daa07e6e6e5e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:15.507078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bc20ea8e-c0fa-48c1-9cd4-daa07e6e6e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:15.613507 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 549/1085] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:15.909228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:15.911422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:15.945557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:16.138417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7c13-e24d-417e-9969-94f4d0884ba9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:16.258696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:16.287416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:16.288497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:16.288782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:16.289066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:16.292054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:16.397332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:16.399003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6ebc9-9c6c-491e-9e49-5c18aa13792e service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:16.419949 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 537/1086] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:25.612250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:26.417313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=54 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:29.290372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255db61-f0d0-447d-93d8-faf513e6f5a1 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:29.291066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255db61-f0d0-447d-93d8-faf513e6f5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:29.291066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255db61-f0d0-447d-93d8-faf513e6f5a1 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:29.292954 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 550/1087] 10.222.0.26 () {54 vars in 918 bytes} [Sun Oct 30 01:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:29.300035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec9d4-7c39-4355-9a73-e58ea91603e6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:29.300713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec9d4-7c39-4355-9a73-e58ea91603e6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:29.300983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec9d4-7c39-4355-9a73-e58ea91603e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:29.399385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d52ec9d4-7c39-4355-9a73-e58ea91603e6 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:29.400799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d52ec9d4-7c39-4355-9a73-e58ea91603e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:29.522667 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 538/1088] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Oct 30 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:29.537236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64020b8c-400a-4465-8df8-3c4863a28e24 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:29.537521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64020b8c-400a-4465-8df8-3c4863a28e24 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:29.537826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64020b8c-400a-4465-8df8-3c4863a28e24 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:29.539237 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 551/1089] 10.222.0.26 () {54 vars in 918 bytes} [Sun Oct 30 01:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:29.544289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dfbf-2abc-408b-b4f1-cd3e016b3083 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:29.544520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dfbf-2abc-408b-b4f1-cd3e016b3083 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:29.544801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dfbf-2abc-408b-b4f1-cd3e016b3083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:29.620797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:29.623517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fc74dfbf-2abc-408b-b4f1-cd3e016b3083 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:29.625306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fc74dfbf-2abc-408b-b4f1-cd3e016b3083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:29.745151 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 539/1090] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Oct 30 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:32.415631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:32.416193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:32.416193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:32.417514 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 552/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:32.423136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f847e4-0c5c-4b23-8ed8-839b1455d908 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:32.423627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f847e4-0c5c-4b23-8ed8-839b1455d908 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:32.423733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f847e4-0c5c-4b23-8ed8-839b1455d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:32.495247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f1f847e4-0c5c-4b23-8ed8-839b1455d908 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:32.496434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f1f847e4-0c5c-4b23-8ed8-839b1455d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:32.555430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:32.603170 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 540/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:32.892672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:32.894187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:33.048417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265605e-ff5c-45b3-a7e5-aa99036f953f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:33.129724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:33.170038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:33.171168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:33.171448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:33.171749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:33.173363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:33.272201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:33.273873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79c247-44ff-46d7-9487-1d7b168cd39b service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:33.292517 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 553/1093] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:34.462504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:34.463105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:34.463105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:34.464462 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 541/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:34.470113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602591fa-19c7-4713-9eae-841921df1c27 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:34.470584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602591fa-19c7-4713-9eae-841921df1c27 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:34.470712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602591fa-19c7-4713-9eae-841921df1c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:34.552147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-602591fa-19c7-4713-9eae-841921df1c27 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:34.553355 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-602591fa-19c7-4713-9eae-841921df1c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:34.593662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:34.690120 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 554/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:02:34.847078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:34.848658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:34.996642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b674-7b53-4e8b-960a-4b08573319ac None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:35.037754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:35.112316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:35.112828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:35.113105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:35.113372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:35.114967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:35.203645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:35.205251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e900266-11bd-40fe-801f-de3585a11c1e service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:35.223680 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 542/1096] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:36.356149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:36.356756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:36.356756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:36.358084 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 555/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:36.363653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4c10-d4d8-4d57-b80d-f4f074f0193f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:36.364116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4c10-d4d8-4d57-b80d-f4f074f0193f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:36.364229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4c10-d4d8-4d57-b80d-f4f074f0193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:36.417771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:36.442743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-280a4c10-d4d8-4d57-b80d-f4f074f0193f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:36.444047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-280a4c10-d4d8-4d57-b80d-f4f074f0193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:36.543336 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 543/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:36.697588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:36.699197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:36.862107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1bd32-b6f3-4d52-8153-20fed11c9839 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:36.875017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:36.989285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:36.989829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:36.990104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:36.990369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:36.992032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:37.082168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:37.083799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1ebf-84eb-483b-b2ba-bb28bac56a59 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:37.101933 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 556/1099] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:38.216700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:38.217273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:38.217273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:38.219163 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 544/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:38.224275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5974-7d26-484e-bef4-29d854acc927 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:38.224733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5974-7d26-484e-bef4-29d854acc927 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:38.224832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5974-7d26-484e-bef4-29d854acc927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:38.244390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:38.303598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ba7d5974-7d26-484e-bef4-29d854acc927 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:38.304908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ba7d5974-7d26-484e-bef4-29d854acc927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:38.404141 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 557/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:38.558311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:38.559872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:38.674177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:38.712159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b82e03-008f-4f8f-8f88-b0728772f5e9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:38.826399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:38.826919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:38.827765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:38.828055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:38.829649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:38.907184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:38.920807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:38.922439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad759fe8-831d-4dd4-8817-98159677284d service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:38.940458 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 545/1102] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:40.052044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:40.052609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:40.052609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:40.053860 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 558/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:40.058869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717398c-f1fb-4cc2-ae07-00a0b78fd0fc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:40.059327 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717398c-f1fb-4cc2-ae07-00a0b78fd0fc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:40.059464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717398c-f1fb-4cc2-ae07-00a0b78fd0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:40.136294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b717398c-f1fb-4cc2-ae07-00a0b78fd0fc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:40.137510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b717398c-f1fb-4cc2-ae07-00a0b78fd0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:40.232018 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 546/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:40.383275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:40.384815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:40.474566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:40.540104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756218-cb6a-4a6d-87d4-ac11b649e322 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:40.659520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:40.660088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:40.660372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:40.660649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:40.662271 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:40.715496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:40.761342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:40.762957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf1e42-2e7a-47aa-9061-883e0922f2bc service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:40.783003 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 559/1105] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:41.939963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:41.940529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:41.940529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:41.941822 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 547/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:41.947158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed303d3-6fcc-49fe-837f-870761b1f029 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:41.947643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed303d3-6fcc-49fe-837f-870761b1f029 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:41.947760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed303d3-6fcc-49fe-837f-870761b1f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:42.025671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0ed303d3-6fcc-49fe-837f-870761b1f029 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:42.026879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0ed303d3-6fcc-49fe-837f-870761b1f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:42.127600 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 560/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:42.280769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:42.282218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:42.330936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:42.430083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c06007-b254-4a74-a4ba-957891895103 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:42.544052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:42.544579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:42.544860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:42.545130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:42.547317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:42.554232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:42.637611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:42.639208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ef395-4cbc-4b87-8533-22615dbec41d service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:42.658088 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 548/1108] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:43.795076 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:43.795674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:43.795674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:43.797541 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 561/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:43.802507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d13e-3d2f-464c-a8a8-18039adcded7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:43.802985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d13e-3d2f-464c-a8a8-18039adcded7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:43.803093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d13e-3d2f-464c-a8a8-18039adcded7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:43.881767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e288d13e-3d2f-464c-a8a8-18039adcded7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:43.882996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e288d13e-3d2f-464c-a8a8-18039adcded7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:43.953104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:43.982208 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 549/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:44.132219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:44.133779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:44.152298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:44.283869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21a8f6-29f1-42c5-b792-5323249aa2ea None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:44.371438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:44.398892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:44.399501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:44.399833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:44.400103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:44.401780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:44.497672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:44.499833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a07e-0ea2-47cb-9a57-c0e6c551b3a2 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:44.519226 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 562/1111] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:45.630561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:45.631168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:45.631168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:45.632396 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 550/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:45.637508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540a0e-94ca-41f0-b0f6-4c81daaa9886 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:45.638037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540a0e-94ca-41f0-b0f6-4c81daaa9886 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:45.638037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540a0e-94ca-41f0-b0f6-4c81daaa9886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:45.718225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4a540a0e-94ca-41f0-b0f6-4c81daaa9886 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:45.721885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4a540a0e-94ca-41f0-b0f6-4c81daaa9886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:45.762768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:45.820256 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 563/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:45.973204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:45.974670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:46.127129 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbe945-1097-4e4a-9dd1-b34ac0528405 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:46.186532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:46.252308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:46.252821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:46.253097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:46.253366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:46.254962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:46.346092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:46.347752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c50120-1f26-4afc-a58c-c263db83210a service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:46.366644 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 551/1114] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:47.552358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:47.553220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:47.553220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:47.554245 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 564/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:47.559553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3ed75-4c02-4b08-b54d-acdbd4c6bddf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:47.560065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3ed75-4c02-4b08-b54d-acdbd4c6bddf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:47.560403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3ed75-4c02-4b08-b54d-acdbd4c6bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:47.642517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-49c3ed75-4c02-4b08-b54d-acdbd4c6bddf None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:47.643685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-49c3ed75-4c02-4b08-b54d-acdbd4c6bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:47.645832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:47.754816 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 552/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:47.906607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:47.908066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:48.062480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa644-eff1-4788-a7c7-5b1fb9c786dc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:48.091389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:48.182796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:48.183318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:48.183620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:48.183900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:48.185517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:48.279131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:48.281309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f20a8-636b-4c6c-ac94-fa4f436d0eea service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:48.301234 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 565/1117] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:02:50.435412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:50.436316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:50.436316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:50.438490 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 553/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:50.445426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552d589-e4d5-4f5d-8e03-de6220feb97c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:50.445995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552d589-e4d5-4f5d-8e03-de6220feb97c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:50.446384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552d589-e4d5-4f5d-8e03-de6220feb97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:50.493160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:50.525224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6552d589-e4d5-4f5d-8e03-de6220feb97c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:50.526496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6552d589-e4d5-4f5d-8e03-de6220feb97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:50.644891 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 566/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:50.934570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:50.936187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:51.041502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f253939-3c41-46ad-8213-c19561985a6e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:51.075125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:51.118571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:51.119244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:51.119463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:51.119769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:51.121407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:51.235927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:51.237292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa542-2a24-4f32-b8d2-2baa76d66e04 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:51.256613 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 554/1120] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:53.712178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:53.713011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:53.713171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:53.714697 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 567/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:53.720978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76906910-9756-44ff-b48f-6b43d814176e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:53.721576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76906910-9756-44ff-b48f-6b43d814176e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:53.721759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76906910-9756-44ff-b48f-6b43d814176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:53.799049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-76906910-9756-44ff-b48f-6b43d814176e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:53.800315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-76906910-9756-44ff-b48f-6b43d814176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:53.815241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:53.906804 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 555/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:54.189337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:54.190885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:54.301619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7d284-24a7-4605-98ed-e3c24937a0bc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:54.315845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:54.383026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:54.383686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:54.383971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:54.384249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:54.385900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:54.508760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:54.510458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd174a0a-b471-4959-8a86-54c592f12ba9 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:54.529997 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 568/1123] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:57.169709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:57.170288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:57.170288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:57.172427 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 556/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:02:57.178108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f32d6-9993-4721-8719-391bcc16666e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:57.178617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f32d6-9993-4721-8719-391bcc16666e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:57.178720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f32d6-9993-4721-8719-391bcc16666e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:57.229099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:57.249539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1f2f32d6-9993-4721-8719-391bcc16666e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:57.250774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1f2f32d6-9993-4721-8719-391bcc16666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:57.351490 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 569/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:57.651108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:57.652999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:57.776798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1246cf-2cb8-41f1-a39a-854615851c64 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:57.822976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:57.863474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:57.864174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:57.864448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:57.864734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:57.866506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:57.992814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:57.994296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92d5a6-868b-4e2f-b688-0a999cd606b7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:58.017493 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 557/1126] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:00.389307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:00.389307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:00.389758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:00.390853 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 570/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:00.396479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754dcb-4889-4387-9062-332429db6d76 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:00.396943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754dcb-4889-4387-9062-332429db6d76 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:00.397056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754dcb-4889-4387-9062-332429db6d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:00.464692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-44754dcb-4889-4387-9062-332429db6d76 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:00.465927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-44754dcb-4889-4387-9062-332429db6d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:00.488317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:00.567932 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 558/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:00.855710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:00.857244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:00.979294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3da444-9159-4828-8427-e00ce9a440a7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:01.001644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:01.063066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:01.063720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:01.063998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:01.064268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:01.065874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:01.201312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:01.204697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7cecf-69d9-4b13-9616-29666f7b4166 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:01.238422 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 571/1129] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:03.526941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:03.527511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:03.527511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:03.529358 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 559/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:03.536814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd9af1-7868-49b2-8917-5b83c5ebcb4a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:03.537353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd9af1-7868-49b2-8917-5b83c5ebcb4a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:03.537942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd9af1-7868-49b2-8917-5b83c5ebcb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:03.650130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:03.653160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a1bd9af1-7868-49b2-8917-5b83c5ebcb4a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:03.654448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a1bd9af1-7868-49b2-8917-5b83c5ebcb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:03.772907 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 572/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:04.067741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:04.069548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:04.195608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b2dc1-9b5f-4df1-8a89-3b54904c0cf8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:04.287208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:04.333765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:04.334839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:04.335359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:04.335901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:04.339132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:04.510577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:04.512367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ce32c-0b16-41be-8d7d-41cb1ac8cb31 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:04.534188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 560/1132] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:06.989738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:06.990398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:06.990398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:06.991565 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 573/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:06.997446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c257a-7a85-4bb8-8418-c894253cfa67 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:06.997949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c257a-7a85-4bb8-8418-c894253cfa67 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:06.998076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c257a-7a85-4bb8-8418-c894253cfa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:07.073888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2b3c257a-7a85-4bb8-8418-c894253cfa67 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:07.075347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b3c257a-7a85-4bb8-8418-c894253cfa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:07.111151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:07.189344 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 561/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:07.478270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:07.481086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:07.640698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba1b56-447e-41b2-8949-c6bd8d9d3f40 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:07.677134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:07.729069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:07.729723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:07.730004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:07.730282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:07.732055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:07.854063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:07.855505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae68a3-b2de-4334-9627-01dd8df816b0 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:07.882169 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 574/1135] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:10.189045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:10.189678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:10.189678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:10.191001 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 562/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:10.197222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577a94d-e0d1-431b-8691-b43bee4b21f6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:10.198600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577a94d-e0d1-431b-8691-b43bee4b21f6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:10.198600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577a94d-e0d1-431b-8691-b43bee4b21f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:10.267915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6577a94d-e0d1-431b-8691-b43bee4b21f6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:10.269145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6577a94d-e0d1-431b-8691-b43bee4b21f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:10.277885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:10.371292 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 575/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:10.667847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:10.670392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:10.821461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea318c-4e57-49a7-97a0-ffdc08827c92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:10.879785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:10.899886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:10.900554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:10.900776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:10.901050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:10.902695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:11.018795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:11.020181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580f6b1-a990-4cdb-9792-d4cef5e767ae service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:11.039221 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 563/1138] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:13.304303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:13.304878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:13.304878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:13.306236 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 576/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:13.311657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9db9-33bc-4972-ac93-3042f9a63909 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:13.312073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9db9-33bc-4972-ac93-3042f9a63909 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:13.312720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9db9-33bc-4972-ac93-3042f9a63909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:13.389850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9a2f9db9-33bc-4972-ac93-3042f9a63909 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:13.391275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9a2f9db9-33bc-4972-ac93-3042f9a63909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:13.435480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:13.503101 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 564/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:13.787166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:13.788770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:13.900447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b0a3-dcbc-4373-ada9-78b57623d802 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:13.945207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:13.982895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:13.983515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:13.983831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:13.984142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:13.985831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:14.112529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:14.114013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819828f9-c21c-4a05-b933-50f3a15089aa service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:14.135083 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 577/1141] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:16.369812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:16.370688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:16.370688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:16.374403 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 565/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:16.381885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743ca3-f289-45d4-bdb5-6b40beef03ac None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:16.382431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743ca3-f289-45d4-bdb5-6b40beef03ac None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:16.382733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743ca3-f289-45d4-bdb5-6b40beef03ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:16.459047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4e743ca3-f289-45d4-bdb5-6b40beef03ac None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:16.460315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4e743ca3-f289-45d4-bdb5-6b40beef03ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:16.495844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:16.596989 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 578/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:16.882921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:16.884967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:16.995604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59b66e-5a5e-403f-a421-f2c15b678ca0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:17.065083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:17.071980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:17.072592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:17.072870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:17.073147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:17.074755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:17.186001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:17.187515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8830-e0c1-4cba-a449-f494ed069693 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:17.207346 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 566/1144] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:19.437097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:19.437676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:19.437796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:19.439127 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 579/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:19.445304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164920d7-73fe-4b04-a0c6-bf038ad1b97c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:19.445865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164920d7-73fe-4b04-a0c6-bf038ad1b97c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:19.445865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164920d7-73fe-4b04-a0c6-bf038ad1b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:19.517317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-164920d7-73fe-4b04-a0c6-bf038ad1b97c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:19.518547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-164920d7-73fe-4b04-a0c6-bf038ad1b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:19.569040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:19.619836 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 567/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:19.904360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:19.906793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:20.017166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b511de-1a5b-4f62-b9f6-6f3af297b76e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:20.068441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:20.097631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:20.098310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:20.098626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:20.098937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:20.100813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:20.217341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:20.219355 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1df0b-5264-46db-abc7-9d4bbcfe925d service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:20.239087 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 580/1147] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:29.618214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=22 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:30.236705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=66 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:41.549387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057d1f-1099-4c16-99dd-870e9a1f7d5f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:41.549995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057d1f-1099-4c16-99dd-870e9a1f7d5f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:41.549995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057d1f-1099-4c16-99dd-870e9a1f7d5f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:41.551314 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 568/1148] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:41.556964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2805-50d3-4ec6-8610-08cd045d1871 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:41.557482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2805-50d3-4ec6-8610-08cd045d1871 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:41.557588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2805-50d3-4ec6-8610-08cd045d1871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:41.629986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-02ae2805-50d3-4ec6-8610-08cd045d1871 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:41.631216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-02ae2805-50d3-4ec6-8610-08cd045d1871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:41.733682 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 581/1149] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:42.033026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:42.033313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:42.033595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:42.035063 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 569/1150] 10.222.0.26 () {54 vars in 962 bytes} [Sun Oct 30 01:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:42.040479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e2ed-566e-415a-b5ba-7dbe1b9a1da4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:42.040767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e2ed-566e-415a-b5ba-7dbe1b9a1da4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:42.041054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e2ed-566e-415a-b5ba-7dbe1b9a1da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:42.121287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3fa6e2ed-566e-415a-b5ba-7dbe1b9a1da4 None None] MFA Rules not processed for user `aa1b9f055f6c4def891834a8114b04b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:42.122511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3fa6e2ed-566e-415a-b5ba-7dbe1b9a1da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:42.144676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:42.203594 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 582/1151] 10.222.0.26 () {58 vars in 1059 bytes} [Sun Oct 30 01:03:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:42.211458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:42.213018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:42.322362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e0216-38c4-4eea-9573-ca1783fac132 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:42.397990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:42.398597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:42.398885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:42.399165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:42.400810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:42.455566 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:42.519331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:42.520832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae5c0d-7212-4a11-a895-7b87001ff4cf service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:42.540750 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 570/1152] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:44.848874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:44.849463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:44.849463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:44.850842 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 583/1153] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:44.856425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8267-1622-4777-a524-e9459856001b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:44.856889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8267-1622-4777-a524-e9459856001b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:44.856990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8267-1622-4777-a524-e9459856001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:44.929416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-36bc8267-1622-4777-a524-e9459856001b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:44.930669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-36bc8267-1622-4777-a524-e9459856001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:45.031428 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 571/1154] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:45.309595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:45.311172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:45.445447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd66ce-3174-4bae-bd97-890a84a2fdd7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:45.504127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:45.554704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:45.555331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:45.555700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:45.555982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:45.556632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:45.557729 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:03:45.557729 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:03:45.557729 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:03:45.558895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef58674-a584-4143-9fce-77888d57ec24 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:45.579326 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 584/1155] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Oct 30 01:03:45 2022] GET /identity/v3/projects => generated 2553 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:03:47.855006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:47.855657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:47.855657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:47.856966 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 572/1156] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:47.863486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11319-0369-4b98-b13f-11d8c32978ce None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:47.864166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11319-0369-4b98-b13f-11d8c32978ce None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:47.864166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11319-0369-4b98-b13f-11d8c32978ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:47.941039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-35e11319-0369-4b98-b13f-11d8c32978ce None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:47.942357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-35e11319-0369-4b98-b13f-11d8c32978ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:48.048257 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 585/1157] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:48.350331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:48.353372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:48.372362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:48.558486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf89ee8-c23f-4d3d-81df-d3eae4dee395 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:48.676119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:48.710395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:48.711074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:48.711405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:48.711775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:48.713585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:48.814268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:48.815902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf155a9-100d-430f-91cf-3fe45994b118 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:48.834931 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 573/1158] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:51.277954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:51.278843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:51.278843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:51.280162 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 586/1159] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:03:51.286183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089be8a-464f-41b3-891b-1adaf2bd2843 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:51.286657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089be8a-464f-41b3-891b-1adaf2bd2843 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:51.287372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089be8a-464f-41b3-891b-1adaf2bd2843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:51.378218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1089be8a-464f-41b3-891b-1adaf2bd2843 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:51.379774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1089be8a-464f-41b3-891b-1adaf2bd2843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:51.430615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:51.517131 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 574/1160] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:03:51.812639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:51.814825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:51.839529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:51.930263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37766-7a1c-48e2-9791-8fc3dbbfebcb None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:52.034965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:52.036077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:52.036584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:52.037051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:52.039524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:52.126872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:52.170271 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:52.171743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06488f-85bb-44b6-8546-a684bfde719a service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:52.193244 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 587/1161] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:54.473262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:54.473890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:54.473890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:54.475337 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 575/1162] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:54.481568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46d37-1f1a-416d-ab65-a48933ebceee None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:54.482133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46d37-1f1a-416d-ab65-a48933ebceee None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:54.482133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46d37-1f1a-416d-ab65-a48933ebceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:54.557678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bee46d37-1f1a-416d-ab65-a48933ebceee None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:54.558922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bee46d37-1f1a-416d-ab65-a48933ebceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:54.662485 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 588/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:54.957269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:54.958896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:55.066795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfba6a-5268-4198-92b3-10dc74f7602d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:55.144983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:55.146467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:55.147102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:55.147415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:55.147720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:55.149381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:55.263830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:55.265272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f355935-1fe5-4ac7-b039-5f2c9a397e6a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:55.283784 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 576/1164] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:57.769438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:57.770036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:57.770036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:57.771354 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 589/1165] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:03:57.779114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990652-2630-44b9-9b65-9e40d933e2e9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:57.779561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990652-2630-44b9-9b65-9e40d933e2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:57.779711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990652-2630-44b9-9b65-9e40d933e2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:57.853812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0a990652-2630-44b9-9b65-9e40d933e2e9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:03:57.855077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0a990652-2630-44b9-9b65-9e40d933e2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:57.914023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:57.964550 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 577/1166] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:03:58.278928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:58.280667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:58.320143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:58.400986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899d11-09b9-42df-8d98-026bffab3db6 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:58.485756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:58.486383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:58.486672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:58.486951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:58.488676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:58.601217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:58.637280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:58.638753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6bcb0-9e40-4f87-93d9-c9ed1646c149 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:58.661805 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 590/1167] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:04:01.221557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:01.222204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:01.222204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:01.223576 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 578/1168] 10.222.0.26 () {54 vars in 927 bytes} [Sun Oct 30 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:04:01.229883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a7ef4-4fe9-4196-8681-02078599db82 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:01.230376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a7ef4-4fe9-4196-8681-02078599db82 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:01.230483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a7ef4-4fe9-4196-8681-02078599db82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:01.298779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f03a7ef4-4fe9-4196-8681-02078599db82 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:04:01.300041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f03a7ef4-4fe9-4196-8681-02078599db82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:01.397712 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 591/1169] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Oct 30 01:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:04:01.695941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:04:01.697513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:01.814733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5014d-b432-4408-80ef-b31f0ef55168 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:04:01.891969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:04:01.892588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:01.892871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:01.893150 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:01.894781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:01.903912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:02.022059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:04:02.024489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5ef9-3c15-4bec-b5a8-ce3afe4e498d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:04:02.048672 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 579/1170] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:04:11.396135 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=82 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:12.045691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=50 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:25.733902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.734485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.734485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.813752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:25.814969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:25.914803 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 592/1171] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:25.923412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.924071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.924071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:26.000839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:26.001979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:26.100978 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 580/1172] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:26.105607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:26.106849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:26.118338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f020ea-ea01-4ce1-8de4-153d3393fd77 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:26.182461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:26.217694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:26.218307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:26.218586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:26.218863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:26.220082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:05:26.220389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:26.221409 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:05:26.221409 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:05:26.221409 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:05:26.222338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:26.234395 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 593/1173] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:05:26.244402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:26.253873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ae11e-30af-4f3d-87ac-b15556efa992 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:26.317539 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:26.352454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:26.353066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:26.353351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:26.353626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:26.354276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:26.355382 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:05:26.355382 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 01:05:26.355382 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 01:05:26.356626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:26.501076 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 581/1174] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:05:26 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:05:26.505496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:26.506969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:26.544286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053f1d-8e37-4ec8-8549-761544f8c5ad admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:26.643831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:26.644449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:26.644731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:26.645006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:26.645659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:26.646688 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:05:26.646688 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:05:26.646688 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:05:26.647942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:26.782227 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 594/1175] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:05:26 2022] POST /identity/v3/users => generated 290 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:05:26.786789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:26.789069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:26.857944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1efb6d-65a8-4604-9b74-8c05dac8f820 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:26.944386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:26.955096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:26.960053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:26.960053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:26.960053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:26.960053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:26.960053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:26.972562 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 582/1176] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:26.976411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:26.977927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:26.989255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc18fb5-a8a7-4713-b8ae-7aae163bb4bc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:27.015018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:27.086120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:27.086739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.087099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.087288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.087984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:27.089588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba12b-d0b1-4ff8-898d-6401dbd0c0b5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:27.102764 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 595/1177] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:27.107368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:27.108946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:27.142943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c670693-9a9b-4f97-b5f0-a6288a1a03dd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:27.239494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:27.240187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.240471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.240750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] PATH_INFO: `/v3/projects/39aa911640f74e16b4e9ce6b878088ec/users/503ed08d9fd14276a42304234533be84/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.264277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39aa911640f74e16b4e9ce6b878088ec, user_id=503ed08d9fd14276a42304234533be84, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:27.267679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a7acd-b831-4c1e-8a23-c4561039917f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:27.362682 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 583/1178] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:05:27 2022] PUT /identity/v3/projects/39aa911640f74e16b4e9ce6b878088ec/users/503ed08d9fd14276a42304234533be84/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:27.374191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa08b0-4ae0-45fe-9109-cf5d87c7b3b2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.374479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa08b0-4ae0-45fe-9109-cf5d87c7b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.374784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa08b0-4ae0-45fe-9109-cf5d87c7b3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.433194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bafa08b0-4ae0-45fe-9109-cf5d87c7b3b2 None None] MFA Rules not processed for user `503ed08d9fd14276a42304234533be84`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:27.434403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bafa08b0-4ae0-45fe-9109-cf5d87c7b3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:27.514424 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 596/1179] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:27.528595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.528867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.529234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.530460 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 584/1180] 10.222.0.26 () {54 vars in 963 bytes} [Sun Oct 30 01:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:05:27.535885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656c62-9d14-4638-93b7-912c74fbf319 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.536150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656c62-9d14-4638-93b7-912c74fbf319 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.536428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656c62-9d14-4638-93b7-912c74fbf319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.567903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:27.617573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-42656c62-9d14-4638-93b7-912c74fbf319 None None] MFA Rules not processed for user `5de8280cded549c58eeabd4e4dc37964`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:27.618804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-42656c62-9d14-4638-93b7-912c74fbf319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:27.699079 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 597/1181] 10.222.0.26 () {58 vars in 1060 bytes} [Sun Oct 30 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:27.707466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:27.709016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:27.758527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:27.812388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4989414-4eed-4fe5-96c1-ecd49409c971 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:27.886224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:27.886828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:27.887104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:27.887372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:27.888987 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:27.988930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:27.990415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:27.994005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:28.009391 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 585/1182] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:28.056432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408fbd1-0675-44e9-ae33-f45ee8ba3c53 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:28.056907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408fbd1-0675-44e9-ae33-f45ee8ba3c53 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:28.057012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408fbd1-0675-44e9-ae33-f45ee8ba3c53 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:28.058282 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 598/1183] 10.222.0.26 () {52 vars in 830 bytes} [Sun Oct 30 01:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:05:28.061977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:28.063768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:28.164466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7a99-654e-4f7b-a77f-805ce882f1e3 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:28.260758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:28.261390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:28.261667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:28.261937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] PATH_INFO: `/v3/users/503ed08d9fd14276a42304234533be84` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:28.273938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503ed08d9fd14276a42304234533be84)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:28.274983 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:05:28.274983 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 01:05:28.274983 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 01:05:28.276237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:28.287228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:28.446747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503ed08d9fd14276a42304234533be84'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:28.447195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503ed08d9fd14276a42304234533be84'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:28.463222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503ed08d9fd14276a42304234533be84'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:28.463845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503ed08d9fd14276a42304234533be84'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:28.489144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503ed08d9fd14276a42304234533be84'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:28.566941 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 586/1184] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:05:28 2022] DELETE /identity/v3/users/503ed08d9fd14276a42304234533be84 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:05:28.571285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:28.571573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:28.571986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:28.632669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:28.634247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:28.710896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:28.723270 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 599/1185] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:28.736287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:28.736287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:28.843055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dd7d-d847-44fa-9798-9a829cee7ca5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:28.898298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:28.917734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:28.918355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:28.918637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:28.918910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:28.920577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:29.035766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:29.037151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafb9d8-8e1c-4213-9b5b-087b3dc5f3e7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:29.056190 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 587/1186] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:29.524122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:29.531769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:29.537329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d421b3-3ffa-477e-9b22-5cce6ab405d7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:29.643844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:29.644483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:29.644765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:29.645042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] PATH_INFO: `/v3/projects/39aa911640f74e16b4e9ce6b878088ec` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:29.655619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39aa911640f74e16b4e9ce6b878088ec)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:29.656698 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:05:29.656698 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:05:29.656698 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:05:29.657921 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:29.773600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39aa911640f74e16b4e9ce6b878088ec', 'request_id': 'req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:29.781509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39aa911640f74e16b4e9ce6b878088ec', 'request_id': 'req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:29.800409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39aa911640f74e16b4e9ce6b878088ec', 'request_id': 'req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:29.883402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] The token cache is being invalidate because project 39aa911640f74e16b4e9ce6b878088ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:05:29.883739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:29.884289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d8a5f1a9-38ea-413e-bca2-c77b27d4e49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:29.885646 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 600/1187] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:05:29 2022] DELETE /identity/v3/projects/39aa911640f74e16b4e9ce6b878088ec => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:05:39.053894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=62 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:39.881026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:04.966900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:04.967500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:04.967500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:04.968783 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 588/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:04.974528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad800b-664c-47f5-8ba5-40a9f2bea63b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:04.975036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad800b-664c-47f5-8ba5-40a9f2bea63b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:04.975146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad800b-664c-47f5-8ba5-40a9f2bea63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:05.060507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c6ad800b-664c-47f5-8ba5-40a9f2bea63b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:05.061748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c6ad800b-664c-47f5-8ba5-40a9f2bea63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:05.174132 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 601/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:05.331380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:05.333712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:05.459920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e67f9c-0c35-4e42-b207-9405d7d6df68 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:05.465690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:05.466325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:05.466615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:05.466899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:05.477204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:05.479230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bc8ac-96b5-4265-8130-43fa903697e5 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:05.490566 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 589/1190] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:07:06.446077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c90a-5029-419e-af74-03d0b6e7faf1 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.446784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c90a-5029-419e-af74-03d0b6e7faf1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.446784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c90a-5029-419e-af74-03d0b6e7faf1 None None] PATH_INFO: `/v3` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.448021 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 602/1191] 10.222.0.61 () {54 vars in 925 bytes} [Sun Oct 30 01:07:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:07:06.454794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.455279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.455378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.526184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:06.527533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:06.601434 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32db-037b-4f22-9df0-f05530e3c8bd None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.601773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32db-037b-4f22-9df0-f05530e3c8bd None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.602008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32db-037b-4f22-9df0-f05530e3c8bd None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.603686 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 603/1192] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:07:06.608223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b008d-df61-4a4d-8ee9-d8afac57aa2d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.608510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b008d-df61-4a4d-8ee9-d8afac57aa2d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.608812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b008d-df61-4a4d-8ee9-d8afac57aa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.611985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:06.617458 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 590/1193] 10.222.0.61 () {58 vars in 1015 bytes} [Sun Oct 30 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:06.632511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5c2b008d-df61-4a4d-8ee9-d8afac57aa2d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:06.633753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5c2b008d-df61-4a4d-8ee9-d8afac57aa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:06.644105 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 604/1194] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:07:06.798300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:06.810536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd556d2-c6bd-4076-8af5-1c1daf3cdb87 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:06.816647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:06.817266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.817550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.817842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.826438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:06.828115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06e104-5206-46b7-bca7-ff8c882d8f69 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:06.838483 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 591/1195] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:15.792353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fcbca-9246-4835-afa2-eb4d3af438f5 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:15.792867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fcbca-9246-4835-afa2-eb4d3af438f5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:15.792979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fcbca-9246-4835-afa2-eb4d3af438f5 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:15.794340 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 605/1196] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:15.799286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951bd8-f294-4791-ab6c-8d10d71c21b2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:15.800311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951bd8-f294-4791-ab6c-8d10d71c21b2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:15.800525 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951bd8-f294-4791-ab6c-8d10d71c21b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:15.820555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9e951bd8-f294-4791-ab6c-8d10d71c21b2 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:15.821749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9e951bd8-f294-4791-ab6c-8d10d71c21b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:15.830188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 592/1197] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:07:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:15.839207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fcbca-9246-4835-afa2-eb4d3af438f5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:15.854815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fcbca-9246-4835-afa2-eb4d3af438f5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:15.861116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:15.861729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:15.862011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:15.862897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:15.871823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:15.873422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52be61-b83e-4d12-a05f-372bf5cb0ced service placement] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:15.883438 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 606/1198] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:18.319430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c7996-a7f6-494b-8c38-93e8d5d93681 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:18.320080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c7996-a7f6-494b-8c38-93e8d5d93681 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:18.320080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c7996-a7f6-494b-8c38-93e8d5d93681 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:18.321392 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 593/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:18.326274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132e4c-dc8a-4da7-b597-bb8f3d4469ce None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:18.326710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132e4c-dc8a-4da7-b597-bb8f3d4469ce None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:18.326833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132e4c-dc8a-4da7-b597-bb8f3d4469ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:18.345830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ed132e4c-dc8a-4da7-b597-bb8f3d4469ce None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:18.347008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ed132e4c-dc8a-4da7-b597-bb8f3d4469ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:18.354837 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 607/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:18.509169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c7996-a7f6-494b-8c38-93e8d5d93681 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:18.523512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c7996-a7f6-494b-8c38-93e8d5d93681 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:18.529084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:18.529681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:18.529962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:18.530237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:18.539646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:18.541329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d626a-f85a-45cd-ae35-77ac6c34e330 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:18.552222 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 594/1201] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:19.630607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1916a-d22d-4c9a-b77c-cdd82fd50b07 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:19.631178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1916a-d22d-4c9a-b77c-cdd82fd50b07 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:19.631178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1916a-d22d-4c9a-b77c-cdd82fd50b07 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:19.632561 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 608/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 01:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:19.637543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306347be-35ff-4ecf-ae4f-4d12f7d4eb2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:19.638075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306347be-35ff-4ecf-ae4f-4d12f7d4eb2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:19.638075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306347be-35ff-4ecf-ae4f-4d12f7d4eb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:19.657982 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-306347be-35ff-4ecf-ae4f-4d12f7d4eb2d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:19.659794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-306347be-35ff-4ecf-ae4f-4d12f7d4eb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:19.671077 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 595/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 01:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:19.822169 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1916a-d22d-4c9a-b77c-cdd82fd50b07 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:19.836692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1916a-d22d-4c9a-b77c-cdd82fd50b07 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:19.842144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:19.842742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:19.843048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:19.843338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:19.852261 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:19.853847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8256b-465b-4d45-bf5c-dfe36a94bd7c service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:19.863477 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 609/1204] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:23.839211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:23.839803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:23.839803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:23.841128 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 596/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:23.846982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62e637-a82b-4582-92a5-f6a719693208 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:23.847426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62e637-a82b-4582-92a5-f6a719693208 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:23.847528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62e637-a82b-4582-92a5-f6a719693208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:23.867134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ef62e637-a82b-4582-92a5-f6a719693208 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:23.868341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ef62e637-a82b-4582-92a5-f6a719693208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:23.876080 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 610/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:24.160856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:24.162597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:24.240249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dedc6-e266-4973-9dee-09f113d8323d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:24.243823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:24.244422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:24.244702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:24.244972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:24.254361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:24.256010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8638-2627-42bb-afae-f40b5af60d02 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:24.265865 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 597/1207] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:26.433387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:26.433829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:26.433961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:26.435305 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 611/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:26.441011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261a53-3d6a-4c27-b6b5-99df2c7ab2fd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:26.441423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261a53-3d6a-4c27-b6b5-99df2c7ab2fd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:26.441564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261a53-3d6a-4c27-b6b5-99df2c7ab2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:26.461287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9b261a53-3d6a-4c27-b6b5-99df2c7ab2fd None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:26.462499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9b261a53-3d6a-4c27-b6b5-99df2c7ab2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:26.470359 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 598/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:26.760167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:26.761900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:26.774928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec034a72-ee7a-44dd-98bd-4a5bfee8abe1 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:26.778630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:26.779239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:26.779524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:26.779829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:26.789477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:26.790934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcfe09-c9b8-476b-801d-4cd4de3b3e07 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:26.801210 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 612/1210] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:30.464935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:30.465516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:30.465516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:30.466878 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 599/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:30.472695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb88f3-e3ef-4fd9-b6c7-26b456ad1beb None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:30.473105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb88f3-e3ef-4fd9-b6c7-26b456ad1beb None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:30.473254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb88f3-e3ef-4fd9-b6c7-26b456ad1beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:30.480802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:30.493206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2eeb88f3-e3ef-4fd9-b6c7-26b456ad1beb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:30.494411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2eeb88f3-e3ef-4fd9-b6c7-26b456ad1beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:30.502322 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 613/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:30.805740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:30.807437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:30.886605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd0142-ac82-444d-ac98-61a8a53391f7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:30.890190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:30.890920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:30.891207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:30.891480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:30.901422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:30.902838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04094c5c-a0d2-43b3-9fba-5042ca370f6c service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:30.913011 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 600/1213] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:33.072814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:33.073343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:33.073343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:33.074634 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 614/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 01:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:07:33.080176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebfc05-c5e6-4e16-90ed-904aaf593426 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:33.080635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebfc05-c5e6-4e16-90ed-904aaf593426 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:33.080859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebfc05-c5e6-4e16-90ed-904aaf593426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:33.101513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-17ebfc05-c5e6-4e16-90ed-904aaf593426 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:33.102719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-17ebfc05-c5e6-4e16-90ed-904aaf593426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:33.110557 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 601/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:33.410590 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:33.412288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:33.426020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e870-196c-47ba-8f73-db1cf62d8923 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:33.429585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:33.430186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:33.430478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:33.430757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:33.440672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:33.442144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc2549-14c6-401b-8903-8f9e52aa8f97 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:33.452333 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 615/1216] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:43.096708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:43.447993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=8 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:01.304066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936a18f-9ecc-4db8-9b85-ba6741ab08f7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.304815 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936a18f-9ecc-4db8-9b85-ba6741ab08f7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.304815 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936a18f-9ecc-4db8-9b85-ba6741ab08f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.339981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e936a18f-9ecc-4db8-9b85-ba6741ab08f7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.342042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e936a18f-9ecc-4db8-9b85-ba6741ab08f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.356555 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 602/1217] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:01.368258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.368864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.368864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.397013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.398298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.407141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ccab6-4b84-42aa-b84a-14ea63b50258 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.407477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ccab6-4b84-42aa-b84a-14ea63b50258 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.407726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ccab6-4b84-42aa-b84a-14ea63b50258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.418403 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 616/1218] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.419934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.435181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb0f0b-0055-479c-a66f-50507b1f2d69 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.442323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.442965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.443251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.443532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.444359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:01.444671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.445432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-854ccab6-4b84-42aa-b84a-14ea63b50258 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.446327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabd7f8-a46f-45e1-bc88-27380b0b01fb admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.451694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-854ccab6-4b84-42aa-b84a-14ea63b50258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.465739 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 617/1219] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:09:01.466931 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 603/1220] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 30 01:09:01.467131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b7f78-48c3-489a-8766-2fe10b866e11 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.467383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b7f78-48c3-489a-8766-2fe10b866e11 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.467790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b7f78-48c3-489a-8766-2fe10b866e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.469102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.469615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.470116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.506108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.507353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.518985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-626b7f78-48c3-489a-8766-2fe10b866e11 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.520257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-626b7f78-48c3-489a-8766-2fe10b866e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.527281 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 604/1221] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.528413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.550187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bee019-f82d-4917-b2ec-b96c85853d78 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.550420 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 618/1222] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.552012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479b6da-0d15-4f6f-87d5-53495be3812a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.552012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479b6da-0d15-4f6f-87d5-53495be3812a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.552225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479b6da-0d15-4f6f-87d5-53495be3812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.564481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.565475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.566592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.567096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.568008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.570394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.581767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4479b6da-0d15-4f6f-87d5-53495be3812a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.582988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4479b6da-0d15-4f6f-87d5-53495be3812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.593569 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 619/1223] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.594518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9386c36-3a2d-42b0-b2bc-fe7f118226f6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.594811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9386c36-3a2d-42b0-b2bc-fe7f118226f6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.595242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9386c36-3a2d-42b0-b2bc-fe7f118226f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.620714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c9386c36-3a2d-42b0-b2bc-fe7f118226f6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.625164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c9386c36-3a2d-42b0-b2bc-fe7f118226f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.630545 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 620/1224] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.631809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e353f7-47d5-4907-a865-da66bdadb21e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.632097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e353f7-47d5-4907-a865-da66bdadb21e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.632375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e353f7-47d5-4907-a865-da66bdadb21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.655154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e0e353f7-47d5-4907-a865-da66bdadb21e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.656397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e0e353f7-47d5-4907-a865-da66bdadb21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.664856 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 621/1225] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.666663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477338cf-0dfa-4a3d-b7de-632fbb4e10ab None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.666959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477338cf-0dfa-4a3d-b7de-632fbb4e10ab None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.667238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477338cf-0dfa-4a3d-b7de-632fbb4e10ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.686875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-477338cf-0dfa-4a3d-b7de-632fbb4e10ab None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.688116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-477338cf-0dfa-4a3d-b7de-632fbb4e10ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.696052 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 622/1226] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.697251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3c829-3b53-4be1-8d3a-26581313417c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.697543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3c829-3b53-4be1-8d3a-26581313417c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.697824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3c829-3b53-4be1-8d3a-26581313417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.717597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-afc3c829-3b53-4be1-8d3a-26581313417c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.718837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-afc3c829-3b53-4be1-8d3a-26581313417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.722742 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 605/1227] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:01.724385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.726877 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 623/1228] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.731456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921f057-1b0f-4283-a7d8-8c85d18db1f4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.731456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921f057-1b0f-4283-a7d8-8c85d18db1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.731456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921f057-1b0f-4283-a7d8-8c85d18db1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.751485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0921f057-1b0f-4283-a7d8-8c85d18db1f4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.753154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0921f057-1b0f-4283-a7d8-8c85d18db1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.793545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4d3d8-3bd6-43ac-a847-c87a15b60599 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.798259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.798906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.799190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.799468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.800158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:01.804857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.804857 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:09:01.804857 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 01:09:01.804857 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 01:09:01.804857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.815081 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 606/1229] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:01.819721 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 624/1230] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.819912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.820004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230020ed-50f5-4754-b8e0-8cb2c6caf07b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.820004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230020ed-50f5-4754-b8e0-8cb2c6caf07b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.820004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230020ed-50f5-4754-b8e0-8cb2c6caf07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.831450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aa0b0-d693-45fa-bbd4-244ce56bcd71 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.836401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.837009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.837294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.837655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.838916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:01.839223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.840893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf0089-521d-41a1-8a5e-6f21e1743413 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.842655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-230020ed-50f5-4754-b8e0-8cb2c6caf07b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.844058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-230020ed-50f5-4754-b8e0-8cb2c6caf07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.852704 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 625/1231] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.860468 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 607/1232] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:01.860656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.860656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.860855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.860953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.861059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.861059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.878219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.878483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:01.879420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.879536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:01.888091 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 626/1233] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.888492 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 608/1234] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:01.889121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.890326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.902138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00265c-ae81-4cb7-9702-6b445145e659 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.903061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8302bdc-ec49-4cb5-b5db-ab0824b8264b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.906808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.907421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.907563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.907795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.908006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.908230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.908513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.908664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:01.908774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.908973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.909402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.910578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.911345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9e130-4e0b-4127-a5c1-1654819319ea admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:01.922733 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 609/1235] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:01.923724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:01.933276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575398-dbf5-4d44-ba26-e6d81fda94e1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:01.937985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:01.938575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:01.938858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:01.939134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:01.939792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:01.941679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.026739 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 627/1236] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.028252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b9ec0-edb9-4747-a004-8405a9a0389a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.028557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b9ec0-edb9-4747-a004-8405a9a0389a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.028837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b9ec0-edb9-4747-a004-8405a9a0389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.047928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-207b9ec0-edb9-4747-a004-8405a9a0389a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:02.057460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-207b9ec0-edb9-4747-a004-8405a9a0389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:02.057596 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 610/1237] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:01 2022] POST /identity/v3/projects => generated 357 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.057596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.060854 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 628/1238] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:02.062099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.062391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.062676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.082488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:02.083730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:02.105899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:02.118925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1527-9d14-4259-bfa1-e78af8085080 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.123736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.124359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.124643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.124924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.125561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.127775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.147178 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 629/1239] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:02.148479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.157384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:02.161409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800e898-cccf-409e-8c1d-a642cecbc80e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.166024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.166679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.166962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.167238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.167965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:02.168273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.169891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.181507 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 630/1240] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:02.182900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.192765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e82617-dd38-4c84-b1e4-882145f43745 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.197949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.198534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.198814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.199089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.199757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:02.200065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.201690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.213612 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 631/1241] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:02.214517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.224030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b217ec-c651-4588-bcbc-13b18285e6c9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.228621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.229203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.229534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.229812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.230454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.231743 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:09:02.231743 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:09:02.231743 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:09:02.232952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.244587 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 611/1242] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.245667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.308708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12951-e203-46af-822e-862905e6ccbe admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.313607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.314216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.314503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.314782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.315431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.317072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.327380 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 612/1243] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:02.328408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.337451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33cb47-91bb-45bd-a82b-ccc4c7b8edae admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.342246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.342844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.343130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.343422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.344071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.345073 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:09:02.345073 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 01:09:02.345073 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 01:09:02.346303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.346841 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 632/1244] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.347865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.412518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffa464-9763-48a9-8f9e-69087a985a19 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.416850 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 613/1245] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/users => generated 279 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.416850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.417648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.418286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.418565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.418840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.419493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:02.420452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.421623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.424820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b410-ccae-47f3-9ea4-9f28f27f1beb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.429498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.430091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.430377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.430663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.431311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:02.432178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.433779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.441320 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 633/1246] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:02.441320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.449044 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 614/1247] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:02.449264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.452590 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417d0f-f673-43be-b975-bf45b5c9e99b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.458745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.459496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.459769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.460163 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.460835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.462831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.464521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da8f78-0873-470f-8a07-757ebd3a0929 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.469134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.469731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.470017 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.470292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.470920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.473900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.569532 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 634/1248] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.570490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.581609 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 615/1249] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.582446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.638816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae9b93-fa44-42f5-b7b1-0af13d13250d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.643892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.644506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.644786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.645060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.645699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.646959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1208c-6533-4bb1-8f98-be18f16a0fde admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.647648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.652153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.652774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.653081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.653360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.654014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.655622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.669782 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 616/1250] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:02.669782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.678209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58179b17-6eeb-4c40-b1da-fb9e04f53a6a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.683928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.684527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.684810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.685085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.685714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.687691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.699016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:02.711177 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 635/1251] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.712258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.722088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37879-8135-432e-9628-439fbbb90fd9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.726660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.732930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.732930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.732930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.732930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.732930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.741131 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 636/1252] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:02.742215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.752852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab8047-4937-4682-a155-93d7ed74d4fa admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.757459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.758036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.758317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.758591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.759205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.761194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.765691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:02.779455 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 617/1253] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.780306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.790603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309edd-bf51-4635-a9a2-8ca2b114cccb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.796199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.802177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.802177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.802177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.802177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.802177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:02.882739 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 637/1254] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.883560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.914938 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 618/1255] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:02.914938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:02.947247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4975a4a-84c8-443c-88f0-a5916c6c642b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.976145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f037053-fd40-4b02-92b4-aec3d215f2e2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:02.981849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:02.982457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:02.982734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:02.983004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:02.983661 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:02.985583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.004662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.008338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.008338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.008338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.008338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.008730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.082231 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 619/1256] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:03.083049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.093270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7bfce-14a7-41ef-b73a-9cb3474830dd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.098192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.098725 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 638/1257] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:02 2022] POST /identity/v3/users => generated 303 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:03.098978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.099291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.099417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.099606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] PATH_INFO: `/v3/projects/82fc550e1caf445d8027e8051e0b4d30/users/2cfba439a18a4c8e8fa0021380128447/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.111923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42743-329b-4eda-bf87-050167ff06c3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.116697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.117303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.117442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82fc550e1caf445d8027e8051e0b4d30, user_id=2cfba439a18a4c8e8fa0021380128447, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.117593 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.117874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.118517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.120150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.120507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.130876 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 639/1258] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.131781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.141142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4718091-7de7-4256-a4be-205a2284d7c8 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.148048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.148048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.148048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.148048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.148556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.149012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.160015 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 640/1259] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.160826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.170784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefdca0-67ce-4d62-b6e8-ba3126de1f18 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.175298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.178442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.178442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.178442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.178442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.178782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.188627 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 641/1260] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.189516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.198876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f47433-97e1-4376-8119-d90291726e45 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.202505 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 620/1261] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:03 2022] PUT /identity/v3/projects/82fc550e1caf445d8027e8051e0b4d30/users/2cfba439a18a4c8e8fa0021380128447/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:03.204682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.246876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:03.258626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.259234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.259522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.259858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.260493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.262432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.266864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbdf1-7a3f-447d-a066-4a8f9e404017 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.271515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.272253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.272495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.272776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.273408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.275876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.385882 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 642/1262] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:03 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:03.386724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.397157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c195de-50ac-4dd5-9934-a44b190300a2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.401555 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 621/1263] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:03 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:03.401555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.402744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.403383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.403701 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.403982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.404634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.406247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.409484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d20ff7-e857-4d89-94b0-ef50eca0b81d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.414416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.415024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.415305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.415596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.416265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.417421 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 643/1264] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.417865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.418243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.428555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea72e5-6822-4155-878f-f6cec39c4e1f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.431083 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 622/1265] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.432107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.437137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.437988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.442199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.442199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.442199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.442199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.442698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d5c8-2f70-440b-9c78-7ab6c0a62713 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.447693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.448378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.448657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.448930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] PATH_INFO: `/v3/projects/42a24c56418f48d48ea02785cfced7f4/users/0d769cecaee04e2ab9b9fc22d53003ae/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.453768 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 644/1266] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.454620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.465090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04ab7-b3c0-48e1-997c-e874a05783a2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.465313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a24c56418f48d48ea02785cfced7f4, user_id=0d769cecaee04e2ab9b9fc22d53003ae, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.467834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.470005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.470189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:03.470608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.470888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.471160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.471882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.473469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e80e9-8a91-4f53-9f0d-14de669fecf5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.483952 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 645/1267] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.485362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.497552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.497552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.505217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:03.506443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:03.514799 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 646/1268] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:03.515570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.526017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d85b3-4a00-4199-ad7f-7ccb93624678 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.530749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.531356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.531685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.531945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.532595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.534225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.544970 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 647/1269] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.545631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.555308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b720c-d2ab-43ea-a12a-8d2dc67303a3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.564048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.564048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.564048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.564048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.564455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.564455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.573831 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 648/1270] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.574682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.584596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe219-8bd4-4496-9b16-a225d405e4ea admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.590697 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 623/1271] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:03 2022] PUT /identity/v3/projects/42a24c56418f48d48ea02785cfced7f4/users/0d769cecaee04e2ab9b9fc22d53003ae/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:03.591770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.647196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.648178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.648489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.648768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.649431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.651808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.666236 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 649/1272] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.666236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.670361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113c46a-6fa3-4dcc-8a2d-e26157e8d8e8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.675325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758f0b3-f628-43d8-ae5e-5cdac6e55408 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.675512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.676197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.676481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.676763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.677461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.684653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.684827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.684827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.684827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.684827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] PATH_INFO: `/v3/projects/445041c33bf944d8a7a39281e3aba6c5/users/3e12208f0ceb41c8bb3b69cbd62e78b2/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.691193 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 624/1273] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.692409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.700992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445041c33bf944d8a7a39281e3aba6c5, user_id=3e12208f0ceb41c8bb3b69cbd62e78b2, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.703536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4182ae-dd64-4c1b-98da-a27513030fa8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.703769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.708725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.713654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.713654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.713654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] PATH_INFO: `/v3/projects/95c63194a18f4768abe23f55608ec8c6/users/a4a4230574c846a9b4f7edd240317ee9/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.728481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c63194a18f4768abe23f55608ec8c6, user_id=a4a4230574c846a9b4f7edd240317ee9, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.731326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.801034 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 650/1274] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:03 2022] PUT /identity/v3/projects/445041c33bf944d8a7a39281e3aba6c5/users/3e12208f0ceb41c8bb3b69cbd62e78b2/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:03.802128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.831223 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 625/1275] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:03 2022] PUT /identity/v3/projects/95c63194a18f4768abe23f55608ec8c6/users/a4a4230574c846a9b4f7edd240317ee9/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:03.832318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.856559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:03.860095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9736eb-a7ff-44de-aa3d-b2a8218e0855 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.898189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d1b03-ce48-4fa1-99f6-a0a1d7499db6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.903234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.903887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.904173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.904446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.905102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.906761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.910926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.911544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.911868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.912141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.916928 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 626/1276] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.917753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.921522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.922961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ea286-9a1c-49f1-8662-608db626d88b service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.927361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2a3a3-5a58-4356-960a-cc4b669c0842 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.933452 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 651/1277] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:03.933587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.934212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.934552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.936409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.936528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.936621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.936718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.936718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.937272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:03.951840 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 627/1278] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:03.951840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:03.960024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a430b-9a74-4c8b-b3d6-4e01f0bde134 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:03.965105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:03.969138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:03.969138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:03.969138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] PATH_INFO: `/v3/projects/422b0059789549cea55b9bcdad68680f/users/fa75a17c73cf472ba19d73ac24b9ef7f/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:03.970829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] MFA Rules not processed for user `0d769cecaee04e2ab9b9fc22d53003ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:03.972259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:03.984204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=422b0059789549cea55b9bcdad68680f, user_id=fa75a17c73cf472ba19d73ac24b9ef7f, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:03.986926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e2b28-dd3d-456b-8585-cbad8fb147b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.044377 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 652/1279] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.045217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.055595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e21e-b842-4ca2-9f03-63da0193bfad None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.060649 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 628/1280] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:03 2022] PUT /identity/v3/projects/422b0059789549cea55b9bcdad68680f/users/fa75a17c73cf472ba19d73ac24b9ef7f/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:04.071055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bae085-0429-4c38-8385-32969f2e95be None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.071055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bae085-0429-4c38-8385-32969f2e95be None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.071055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bae085-0429-4c38-8385-32969f2e95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.082160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-41bae085-0429-4c38-8385-32969f2e95be None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.083458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-41bae085-0429-4c38-8385-32969f2e95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:04.105709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:04.114217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.114486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.114766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.115073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] PATH_INFO: `/v3/projects/8519baec3bd94590b0d88c46e6749c91/users/e3b511d83173433582b206bdc3e796db/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.131438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8519baec3bd94590b0d88c46e6749c91, user_id=e3b511d83173433582b206bdc3e796db, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.134168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.147573 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 629/1281] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.148851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.149139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.149416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.168955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.170163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:04.177998 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 630/1282] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.178738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.194008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdb301-2255-4a75-9754-00ddf3730a36 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.198836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.199511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.199825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.200105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] PATH_INFO: `/v3/projects/5bc4fa42794443cc92ffc01b0a27fcdd/users/8b4f2c02f8eb47f58f6953a084568a32/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.215054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc4fa42794443cc92ffc01b0a27fcdd, user_id=8b4f2c02f8eb47f58f6953a084568a32, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.217816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.236290 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 653/1283] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:04 2022] PUT /identity/v3/projects/8519baec3bd94590b0d88c46e6749c91/users/e3b511d83173433582b206bdc3e796db/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:04.237274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.260275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:04.294539 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 631/1284] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:04 2022] PUT /identity/v3/projects/5bc4fa42794443cc92ffc01b0a27fcdd/users/8b4f2c02f8eb47f58f6953a084568a32/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:04.295437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.301464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4ad9-dbd8-400a-88e0-089812527c7c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.358322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.358851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9cf1b-77a1-4e71-99a9-87b4fb2c6e1d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.359054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.359268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.359547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] PATH_INFO: `/v3/projects/f2113ee265c94b28b7475544517787dd/users/e9eb5fda289b42dc93a40914b184d8b8/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.363679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.364287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.364569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.364848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.365478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.367499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae2e9e-2177-4891-9439-641e2b1abf1c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.375074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2113ee265c94b28b7475544517787dd, user_id=e9eb5fda289b42dc93a40914b184d8b8, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.380194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.523797 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 632/1285] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/projects => generated 357 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:04.525068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.525902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.526184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.553777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] MFA Rules not processed for user `fa75a17c73cf472ba19d73ac24b9ef7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.555005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:04.619930 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 633/1286] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.620786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.632560 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 654/1287] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:04 2022] PUT /identity/v3/projects/f2113ee265c94b28b7475544517787dd/users/e9eb5fda289b42dc93a40914b184d8b8/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:04.633304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:04.635528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.684457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0f1ea-aec9-426f-9213-2a2446c0c032 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.706204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede553-3c0d-4b8f-af7a-ddb4f6d188ac admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.710734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.711365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.712607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.712607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.742222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.743366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.744746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.745256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.786745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.788465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabe134-df2c-497b-a394-426258c457bf service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.794330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:04.799953 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 655/1288] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:04.801276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030721-4733-4bdd-808b-4717f071c539 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.801632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030721-4733-4bdd-808b-4717f071c539 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.801978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030721-4733-4bdd-808b-4717f071c539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.821377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.822883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-74030721-4733-4bdd-808b-4717f071c539 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.824111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f31ff-0e6a-4c43-b193-9c938a2bc14f service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.825664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-74030721-4733-4bdd-808b-4717f071c539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:04.836892 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 634/1289] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:04.838223 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 656/1290] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.838374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e3300-72ee-47fb-ab9a-11dc8b550849 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.838765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e3300-72ee-47fb-ab9a-11dc8b550849 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.839051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e3300-72ee-47fb-ab9a-11dc8b550849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.839371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030721-4733-4bdd-808b-4717f071c539 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:04.850216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030721-4733-4bdd-808b-4717f071c539 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:04.856823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:04.857543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.857915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.858456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.859175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:04.861224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a433-cde3-4665-8dc9-eef1956c992d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:04.879954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4a3e3300-72ee-47fb-ab9a-11dc8b550849 None None] MFA Rules not processed for user `8b4f2c02f8eb47f58f6953a084568a32`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.881203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4a3e3300-72ee-47fb-ab9a-11dc8b550849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:04.944480 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 635/1291] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:04.945732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:04.946571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:04.946571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:04.975018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] MFA Rules not processed for user `e9eb5fda289b42dc93a40914b184d8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:04.976171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:05.022245 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 657/1292] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:05.024781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac26ab-735c-4d2d-9617-461da0c0f4e0 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.025280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac26ab-735c-4d2d-9617-461da0c0f4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.025953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac26ab-735c-4d2d-9617-461da0c0f4e0 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.028103 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 658/1293] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:05.038045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac26ab-735c-4d2d-9617-461da0c0f4e0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.053235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac26ab-735c-4d2d-9617-461da0c0f4e0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.059614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.060378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.060759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.061130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.062285 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 636/1294] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:05.063615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.073756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.075811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.097853 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 659/1295] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:05.098828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.109449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2519ea-142e-4a11-8c12-1bcde6313ab6 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.114523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.115309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.116273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.116273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.116617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.118229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c2117-e0b5-4b6f-b3cf-58a2a93583d6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.133328 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 660/1296] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:05.133328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.133328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.133328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.144796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:05.146557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058af48-789b-44ea-84e6-15ab71a10bec None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.150802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.154843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.154843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.154843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.161671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.163366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.171973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:05.174349 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 637/1297] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:05.175228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.186975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:05.187684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dad64a-2563-42fc-8e87-b9a6aad67458 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.193713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.194327 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.194921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.195045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.195951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.198363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.280839 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 638/1298] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:05 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:05.281873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.289914 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 661/1299] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:09:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:05.296507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.315481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bc0d7-1500-4ab2-a39d-3859befc83b6 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.381115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611cf1-e4d3-4fee-b196-56d7b51c9b69 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.387216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.388059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.388343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.388714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] PATH_INFO: `/v3/projects/ba2aebddaf564657919cc25bd8b7b506/users/4600b31dd5b74f01ba6319787411975d/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.407656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2aebddaf564657919cc25bd8b7b506, user_id=4600b31dd5b74f01ba6319787411975d, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.410917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.450563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.451294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.451614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.451935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.452933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.456278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.467912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:05.476741 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 639/1300] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:05 2022] PUT /identity/v3/projects/ba2aebddaf564657919cc25bd8b7b506/users/4600b31dd5b74f01ba6319787411975d/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:05.482455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.594234 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 662/1301] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:05 2022] POST /identity/v3/users => generated 291 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:05.594400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6d221-914e-4bae-969b-b69ec72c9d2d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.594864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.600639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.601283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.601568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.601845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.622019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e9d0-577a-4957-9b8b-3bac3bb17e41 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.652505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:05.690864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.692822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.705972 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 640/1302] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:05.706912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.707859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.708380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.708868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.709747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.711631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.712703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.724628 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 663/1303] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:05.729392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.729560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c310-0887-4281-841d-6dae4e15109d service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.735412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.736011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.736296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.736574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.737225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.742768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a058-a43f-41db-8341-1c04c893f77c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.750275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.756407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.756407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.756407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] PATH_INFO: `/v3/projects/ba2aebddaf564657919cc25bd8b7b506/users/4600b31dd5b74f01ba6319787411975d/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.756407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2aebddaf564657919cc25bd8b7b506, user_id=4600b31dd5b74f01ba6319787411975d, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:05.756794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.758347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba41aa-9a07-478d-b21f-a56911088caa admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:05.773596 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 664/1304] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:05 2022] PUT /identity/v3/projects/ba2aebddaf564657919cc25bd8b7b506/users/4600b31dd5b74f01ba6319787411975d/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:05.784642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.785964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.786414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.796776 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 641/1305] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:05.802386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:05.821468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] MFA Rules not processed for user `4600b31dd5b74f01ba6319787411975d`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:05.823613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437bc0-859a-45bc-a43f-3a6f02a69faf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:05.824034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:05.961446 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 665/1306] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:05.965513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:05.966559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:05.967027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:05.967461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] PATH_INFO: `/v3/projects/2b406decd8664d988dea0473486723c0/users/f8141e60f84948ed8ebfbec8e424adca/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:05.971745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:06.000929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b406decd8664d988dea0473486723c0, user_id=f8141e60f84948ed8ebfbec8e424adca, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.004012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:06.043618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b46e43-397c-4995-8412-20a1c06c1421 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:06.048033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:06.053308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.053308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.053308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.061846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.063864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29299b1c-cee3-4384-974c-2161a269e874 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:06.088522 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 666/1307] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:06.090160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e182ca5-33c6-4198-bbd9-6dcb68ab86e0 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.093257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e182ca5-33c6-4198-bbd9-6dcb68ab86e0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.093694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e182ca5-33c6-4198-bbd9-6dcb68ab86e0 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.097526 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 667/1308] 10.222.0.26 () {54 vars in 912 bytes} [Sun Oct 30 01:09:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:06.105700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.105992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.106414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.135127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:06.136436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:06.165312 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 642/1309] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:05 2022] PUT /identity/v3/projects/2b406decd8664d988dea0473486723c0/users/f8141e60f84948ed8ebfbec8e424adca/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:06.170189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:06.205764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:06.235161 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 668/1310] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:09:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:06.263515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:06.272648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d2e13-9e7e-4708-a433-c9d865cf46ef None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:06.297555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff94af-7dcb-4d0a-9703-873daa8fb968 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:06.304840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:06.306098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.306765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.307174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.308200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.311570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:06.341818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:06.343334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.343853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.344303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.350784 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 643/1311] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:06.352789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:06.369619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dee32-7de4-40c6-ba0f-d7a9c1b50a3c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:06.387465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:06.388331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.388687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.389213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] PATH_INFO: `/v3/projects/2b406decd8664d988dea0473486723c0/users/f8141e60f84948ed8ebfbec8e424adca/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.402420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b406decd8664d988dea0473486723c0, user_id=f8141e60f84948ed8ebfbec8e424adca, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.419660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2080279-c264-4da3-94d8-264e6fdf2902 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:06.439033 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 644/1312] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:06 2022] PUT /identity/v3/projects/2b406decd8664d988dea0473486723c0/users/f8141e60f84948ed8ebfbec8e424adca/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:06.449325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75db6f-8b48-4ddc-9192-5cbefd724298 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.449663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75db6f-8b48-4ddc-9192-5cbefd724298 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.449893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75db6f-8b48-4ddc-9192-5cbefd724298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.503631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.507177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:06.508669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1d75db6f-8b48-4ddc-9192-5cbefd724298 None None] MFA Rules not processed for user `f8141e60f84948ed8ebfbec8e424adca`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:06.512482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1d75db6f-8b48-4ddc-9192-5cbefd724298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:06.517642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:06.524638 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 669/1313] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:06.680010 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 645/1314] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:06.765273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:06.779890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23b47-bcea-4c9e-b86b-ed440f24af03 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:06.903711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:06.909755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:06.909755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:06.909755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:06.993497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:06.996384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:07.021026 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 670/1315] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:07.665617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4b503-a53d-48dc-b79d-42a46b14e577 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:07.665617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4b503-a53d-48dc-b79d-42a46b14e577 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:07.665617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4b503-a53d-48dc-b79d-42a46b14e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:07.716875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:07.732616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded71e1-f7e4-4a81-b778-0887d88f5c8c service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:07.751754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:07.754928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:07.754928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:07.754928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:07.754928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:07.759633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6ed4b503-a53d-48dc-b79d-42a46b14e577 None None] MFA Rules not processed for user `e3b511d83173433582b206bdc3e796db`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:07.772072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ed4b503-a53d-48dc-b79d-42a46b14e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:07.790174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:07.793705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:07.870011 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 671/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:07.917577 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 646/1317] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:07.918586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:08.118889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348566-5593-4047-aa1c-5a2d5cf57f74 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:08.124720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:08.125998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.126450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.126821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.127679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:08.130585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:08.396057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec443b7-c65d-4297-bcf8-123838674933 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.396949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec443b7-c65d-4297-bcf8-123838674933 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.397243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec443b7-c65d-4297-bcf8-123838674933 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.399671 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 647/1318] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:09:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:09:08.409846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a3ec-e1a7-48fe-a653-ac1ad44911d5 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.410133 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a3ec-e1a7-48fe-a653-ac1ad44911d5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.411280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a3ec-e1a7-48fe-a653-ac1ad44911d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.436884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:08.453140 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 672/1319] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:07 2022] POST /identity/v3/projects => generated 405 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:08.455049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bab7a3ec-e1a7-48fe-a653-ac1ad44911d5 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:08.455862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:08.456358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bab7a3ec-e1a7-48fe-a653-ac1ad44911d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:08.613773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf792-329c-48a3-b110-29cda8214254 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:08.614606 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 648/1320] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:08.618733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:08.619947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.620232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.620510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.621155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:08.623194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:08.624541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b179-7305-4470-aa96-f04d799d8760 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.625977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b179-7305-4470-aa96-f04d799d8760 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.626766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b179-7305-4470-aa96-f04d799d8760 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.636204 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 649/1321] 10.222.0.26 () {54 vars in 912 bytes} [Sun Oct 30 01:09:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:09:08.638042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.638774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.639070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.703549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:08.707747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:08.758937 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 673/1322] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:08 2022] POST /identity/v3/users => generated 309 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:08.760456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:08.760792 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 650/1323] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:08.775190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99b1e9-4424-49e1-a61c-4d77a3ad433b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:08.797123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:08.797933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.798505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.798981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.810004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:08.810004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:08.821648 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 674/1324] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:08.823100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:08.841430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc815248-d514-4b02-a4f3-184af2f448a7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:08.847048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:08.847783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:08.848071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:08.848350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] PATH_INFO: `/v3/projects/4408c8a00c184399953640947496996a/users/a10f9523b92a4043aff821768a431c6d/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:08.876216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4408c8a00c184399953640947496996a, user_id=a10f9523b92a4043aff821768a431c6d, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:08.882307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:08.887666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:08.911273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:09.030836 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 651/1325] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:08 2022] PUT /identity/v3/projects/4408c8a00c184399953640947496996a/users/a10f9523b92a4043aff821768a431c6d/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:09.030836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:09.128274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ebec7-b73a-4dac-8c06-089363014365 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:09.148635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc39c2-690a-4e0f-80fa-f5f6fd29b51c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:09.169070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:09.174405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:09.174405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:09.174405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:09.174405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:09.174405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:09.192812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:09.193356 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 652/1326] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:09.196504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:09.197601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:09.197939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:09.198651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:09.219402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36dd31-c3f0-4608-89fb-c4d34ff921bf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:09.226880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:09.228939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:09.229372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:09.229720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] PATH_INFO: `/v3/projects/4408c8a00c184399953640947496996a/users/a10f9523b92a4043aff821768a431c6d/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:09.232301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4408c8a00c184399953640947496996a, user_id=a10f9523b92a4043aff821768a431c6d, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:09.235149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:09.251824 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 653/1327] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:09 2022] PUT /identity/v3/projects/4408c8a00c184399953640947496996a/users/a10f9523b92a4043aff821768a431c6d/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:09.320013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:09.321565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45a53a-0c26-4bdb-b1d6-c5077036bba0 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:09.339785 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 675/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:11.715993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:11.929937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:11.970491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7108-62af-4040-b123-b8a8571ae487 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:11.996498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:11.997178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:11.997465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:11.997727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] PATH_INFO: `/v3/users/fa75a17c73cf472ba19d73ac24b9ef7f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:12.007341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa75a17c73cf472ba19d73ac24b9ef7f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:12.013775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:12.256230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa75a17c73cf472ba19d73ac24b9ef7f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:12.257021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa75a17c73cf472ba19d73ac24b9ef7f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:12.276957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa75a17c73cf472ba19d73ac24b9ef7f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:12.277891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa75a17c73cf472ba19d73ac24b9ef7f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:12.314141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa75a17c73cf472ba19d73ac24b9ef7f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:12.432999 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 654/1329] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:11 2022] DELETE /identity/v3/users/fa75a17c73cf472ba19d73ac24b9ef7f => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:12.465210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46a52-9e35-4c5d-9684-cf7cda4e26ee None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:12.465771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46a52-9e35-4c5d-9684-cf7cda4e26ee None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:12.465771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46a52-9e35-4c5d-9684-cf7cda4e26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:12.517423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a1f46a52-9e35-4c5d-9684-cf7cda4e26ee None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:12.518712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a1f46a52-9e35-4c5d-9684-cf7cda4e26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:12.682128 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 676/1330] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:12.697768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:12.698692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:12.699140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:12.699811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:12.699947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:12.720510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:12.761205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] MFA Rules not processed for user `a10f9523b92a4043aff821768a431c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:12.764837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:12.842681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132d0d-6adb-4e9b-a935-2c467ffae910 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:12.847865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:12.848781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:12.850003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:12.850541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:12.863487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:12.883077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cd084-8e05-46db-981a-b4417e99081a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:12.910447 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 677/1331] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:12.912324 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 655/1332] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:14.199525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:14.201291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:14.224533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe467e-95ff-406f-b421-95aa1680c1a4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:14.236236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:14.236905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.237192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.237471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] PATH_INFO: `/v3/projects/422b0059789549cea55b9bcdad68680f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.245490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] RBAC: Authorizing `identity:delete_project(project_id=422b0059789549cea55b9bcdad68680f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:14.248102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:14.407860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422b0059789549cea55b9bcdad68680f', 'request_id': 'req-6c1f9d86-d0ef-4257-a590-19665de1445f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:14.422064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422b0059789549cea55b9bcdad68680f', 'request_id': 'req-6c1f9d86-d0ef-4257-a590-19665de1445f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:14.436767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422b0059789549cea55b9bcdad68680f', 'request_id': 'req-6c1f9d86-d0ef-4257-a590-19665de1445f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:14.554553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] The token cache is being invalidate because project 422b0059789549cea55b9bcdad68680f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:14.563609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:14.563609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c1f9d86-d0ef-4257-a590-19665de1445f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:14.563609 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 678/1333] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:14 2022] DELETE /identity/v3/projects/422b0059789549cea55b9bcdad68680f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:14.565776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c86a-d280-44ea-985b-cb01363c0017 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.566503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c86a-d280-44ea-985b-cb01363c0017 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.566503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c86a-d280-44ea-985b-cb01363c0017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.604394 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6564c86a-d280-44ea-985b-cb01363c0017 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:14.605739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6564c86a-d280-44ea-985b-cb01363c0017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:14.771639 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 656/1334] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:14.777952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.778482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.778923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.783542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050f9d-6e6c-422e-acfb-bdb0d054d3ad None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.783787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050f9d-6e6c-422e-acfb-bdb0d054d3ad None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.783885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050f9d-6e6c-422e-acfb-bdb0d054d3ad None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.786316 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 657/1335] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 01:09:14.829788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:14.831052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:14.838631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.838929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.839268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.861750 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 679/1336] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:14.867469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:14.868961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:14.886221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:14.887515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:14.898997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bdf39-485c-4bf7-88dd-d095388e4d3e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:14.912513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:14.924708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:14.925679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:14.925830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:14.926060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] PATH_INFO: `/v3/users/e9eb5fda289b42dc93a40914b184d8b8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:14.927732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9eb5fda289b42dc93a40914b184d8b8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:14.928770 dv-cinder-master-3-smb devstack@keystone.service[81636]: WARNING py.warnings [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:09:14.928770 dv-cinder-master-3-smb devstack@keystone.service[81636]: warnings.warn(msg) Oct 30 01:09:14.928770 dv-cinder-master-3-smb devstack@keystone.service[81636]: Oct 30 01:09:14.930015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.088268 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 658/1337] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:09:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:15.089165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.120885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63a49b-38e4-470c-8a98-632e92ca2701 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.139544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.148699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.186303 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 659/1338] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:15.188664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.197804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:15.248678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb9bbf-0d5b-4e23-8763-1b2b1ab55621 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.278898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.279556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.279866 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.280154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.282695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9eb5fda289b42dc93a40914b184d8b8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:15.283611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9eb5fda289b42dc93a40914b184d8b8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:15.298531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.301376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.329408 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 660/1339] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:15.330262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.339597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:15.342700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9eb5fda289b42dc93a40914b184d8b8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:15.357301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54cc4-d48b-403e-9317-d7df3038f604 service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.365716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.366352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.367393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.367393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.378202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.378202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.495542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9eb5fda289b42dc93a40914b184d8b8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:15.514807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26eab557-5fcd-45a4-b85e-18e13773f80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9eb5fda289b42dc93a40914b184d8b8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:15.538876 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 680/1340] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:14 2022] DELETE /identity/v3/users/e9eb5fda289b42dc93a40914b184d8b8 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:15.543240 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 661/1341] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:15 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:15.552440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.557061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:15.561325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.562098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.563203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.668286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:15.680513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:15.790428 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 681/1342] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:15.798444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bd1d-4a08-4382-9044-0679be76dfd8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.821171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.821872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.822194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.822580 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.823333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.826067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.841184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.842946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:15.949913 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 662/1343] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:15 2022] POST /identity/v3/users => generated 288 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:15.952759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:15.956832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ef593-cf28-46db-aa08-e192256fb325 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.964836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.966319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.966777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.967857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.970795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2797b2-1cd5-4200-8153-2b77e87b4adb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:15.985242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.986238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:15.991708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:15.991708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:15.991708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:15.992105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:15.992209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:15.992209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:16.002019 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 682/1344] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:16.013912 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 663/1345] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:16.015869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:16.033384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c11825-4424-40a6-8dd2-f9df4ba63285 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:16.044398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:16.046082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:16.046839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:16.048848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:16.050334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:16.052472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d7dd-41ee-4eb3-807c-05385633d04e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:16.077433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:16.080106 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 683/1346] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:16.091283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195528a-8911-462c-abd4-d608a5361ab6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:16.101755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:16.102620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:16.102758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:16.103048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] PATH_INFO: `/v3/projects/9b823b27bf954e1493a0f580c9edab4f/users/a72943f3d50a43a89118b9a80f34bccf/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:16.143007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b823b27bf954e1493a0f580c9edab4f, user_id=a72943f3d50a43a89118b9a80f34bccf, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:16.145725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:16.248814 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 664/1347] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:16 2022] PUT /identity/v3/projects/9b823b27bf954e1493a0f580c9edab4f/users/a72943f3d50a43a89118b9a80f34bccf/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:16.266647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a745b-8f23-4b87-8f4b-88b465b319bc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:16.267098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a745b-8f23-4b87-8f4b-88b465b319bc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:16.267482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a745b-8f23-4b87-8f4b-88b465b319bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:16.332178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-118a745b-8f23-4b87-8f4b-88b465b319bc None None] MFA Rules not processed for user `a72943f3d50a43a89118b9a80f34bccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:16.334392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-118a745b-8f23-4b87-8f4b-88b465b319bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:16.545602 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 684/1348] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:16.557933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:16.559821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:16.848644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07852-f058-4170-aea5-7c405de823db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:16.857285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:16.858195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:16.858572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:16.858897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:16.874398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:16.879712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:16.894287 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 665/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:17.377199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:17.377199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:17.377199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:17.378650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:17.378650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:17.486395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:17.525277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] MFA Rules not processed for user `a4a4230574c846a9b4f7edd240317ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:17.527677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:17.660459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7af1f-0907-4fab-9e87-8e17a75be94b service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:17.685254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:17.685965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:17.686251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:17.686729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] PATH_INFO: `/v3/projects/f2113ee265c94b28b7475544517787dd` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:17.688362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2113ee265c94b28b7475544517787dd)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:17.689571 dv-cinder-master-3-smb devstack@keystone.service[81635]: WARNING py.warnings [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 01:09:17.689571 dv-cinder-master-3-smb devstack@keystone.service[81635]: warnings.warn(msg) Oct 30 01:09:17.689571 dv-cinder-master-3-smb devstack@keystone.service[81635]: Oct 30 01:09:17.692205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:17.708646 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 685/1350] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:17.710145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:17.738095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3604d2-7ca2-4ace-864c-b8093c509c29 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:17.749659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:17.751464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:17.751768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:17.752254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] PATH_INFO: `/v3/users/a72943f3d50a43a89118b9a80f34bccf` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:17.755388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72943f3d50a43a89118b9a80f34bccf)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:17.758422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:17.804489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:17.889518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2113ee265c94b28b7475544517787dd', 'request_id': 'req-eb2ba03c-17b1-4e97-8786-14f051fb9559'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:17.913761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2113ee265c94b28b7475544517787dd', 'request_id': 'req-eb2ba03c-17b1-4e97-8786-14f051fb9559'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.073680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2113ee265c94b28b7475544517787dd', 'request_id': 'req-eb2ba03c-17b1-4e97-8786-14f051fb9559'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.195366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] The token cache is being invalidate because project f2113ee265c94b28b7475544517787dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:18.198891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72943f3d50a43a89118b9a80f34bccf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.199971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72943f3d50a43a89118b9a80f34bccf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.211963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.220388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-eb2ba03c-17b1-4e97-8786-14f051fb9559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.239406 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 666/1351] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:17 2022] DELETE /identity/v3/projects/f2113ee265c94b28b7475544517787dd => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:18.243116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0672-9592-4a32-9d33-87d2f89ba6cf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:18.248610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0672-9592-4a32-9d33-87d2f89ba6cf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:18.248900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0672-9592-4a32-9d33-87d2f89ba6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:18.270242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72943f3d50a43a89118b9a80f34bccf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.348293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3eae0672-9592-4a32-9d33-87d2f89ba6cf None None] MFA Rules not processed for user `a4a4230574c846a9b4f7edd240317ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:18.349549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3eae0672-9592-4a32-9d33-87d2f89ba6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:18.441705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72943f3d50a43a89118b9a80f34bccf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.471513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72943f3d50a43a89118b9a80f34bccf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:18.497682 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 686/1352] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:17 2022] DELETE /identity/v3/users/a72943f3d50a43a89118b9a80f34bccf => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:18.499044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:18.507366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:18.591039 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 667/1353] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:18.591039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:18.591039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:18.591039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:18.652585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:18.664350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:18.746320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7938e-7532-4c74-b5a0-17c469bec186 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:18.780492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:18.781166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:18.781449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:18.781906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] PATH_INFO: `/v3/users/f8141e60f84948ed8ebfbec8e424adca` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:18.788298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8141e60f84948ed8ebfbec8e424adca)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:18.791009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:18.796209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:19.209074 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 668/1354] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:19.209074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:19.209074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:19.209074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b05e7-27d8-401b-b9d6-75771c9f7a9d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:19.212131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:19.212131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:19.212131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:19.212131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:19.214475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:19.215216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8141e60f84948ed8ebfbec8e424adca'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:19.215216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8141e60f84948ed8ebfbec8e424adca'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:19.215216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8141e60f84948ed8ebfbec8e424adca'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:19.248383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:19.250905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:19.264014 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 669/1355] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:19.264014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:19.264839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:19.329211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8141e60f84948ed8ebfbec8e424adca'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:19.347036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4430216-0690-49eb-9312-a4bf68f12057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8141e60f84948ed8ebfbec8e424adca'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:19.379534 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 687/1356] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:18 2022] DELETE /identity/v3/users/f8141e60f84948ed8ebfbec8e424adca => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:19.385852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:19.389089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:19.390324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:19.396494 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 688/1357] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:09:19 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:19.421981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d46851-b3cf-433c-a2ad-4633151cd6e4 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:19.698142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:19.698970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:19.699396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:19.699879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:19.702065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:19.732130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:19.734090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:19.856651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af09f1-2c90-4d56-a120-e810a3e537df None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:19.863514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:19.864280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:19.865392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36591f22-0c5d-4cf2-ba7e-c5978c473b3d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:19.866305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:19.866880 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:19.867697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] PATH_INFO: `/v3/users/8b4f2c02f8eb47f58f6953a084568a32` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:19.869328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4f2c02f8eb47f58f6953a084568a32)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:19.872141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:19.892734 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 670/1358] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:19.895656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:19.896429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:19.896711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:19.925169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:19.926694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.027273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:20.121341 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 671/1359] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:09:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:20.124572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:20.127316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.159019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4f2c02f8eb47f58f6953a084568a32'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.160026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4f2c02f8eb47f58f6953a084568a32'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.172618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99600d-816c-49a8-b9a5-b974ae32d1bb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:20.186935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:20.191635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:20.192015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:20.194291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] PATH_INFO: `/v3/users/e3b511d83173433582b206bdc3e796db` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:20.198735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b511d83173433582b206bdc3e796db)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:20.202717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4f2c02f8eb47f58f6953a084568a32'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.207374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:20.336929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4f2c02f8eb47f58f6953a084568a32'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.348388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4f2c02f8eb47f58f6953a084568a32'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.370577 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 689/1360] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:19 2022] DELETE /identity/v3/users/8b4f2c02f8eb47f58f6953a084568a32 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:20.375538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:20.377421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.480545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b511d83173433582b206bdc3e796db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.482405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b511d83173433582b206bdc3e796db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.501631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b511d83173433582b206bdc3e796db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.504506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b511d83173433582b206bdc3e796db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.536958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-994ac88f-e2a1-43b9-b46a-e01f6367ff7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b511d83173433582b206bdc3e796db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:20.564710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15bff2-c046-409c-999f-702922705c3f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:20.572415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:20.573207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:20.573667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:20.573954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:20.576466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.599143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:20.601606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:20.625040 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 672/1361] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:20 2022] DELETE /identity/v3/users/e3b511d83173433582b206bdc3e796db => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:20.632765 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 690/1362] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:20.633003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:20.633657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:20.635058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:20.697376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:20.741212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:20.747628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.787422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:20.789831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.925088 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 673/1363] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:20.968892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:20.972486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:20.982753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f13c3e-5789-4e9a-b7c5-78edfb119361 service placement] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:20.999697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:21.001068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.001608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.002861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] PATH_INFO: `/v3/projects/2b406decd8664d988dea0473486723c0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.004874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b406decd8664d988dea0473486723c0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:21.010964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:21.039033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:21.109411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717c9b2-e80b-4f68-aaee-7c3d81387e2c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:21.115173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:21.116694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.117059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.117408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.133159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:21.135480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:21.158675 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 674/1364] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:21.162681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:21.168365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:21.196147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b00b-7032-4015-8aa1-1a5a5ea45fb0 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:21.208742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:21.209412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.210036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.210312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] PATH_INFO: `/v3/projects/9b823b27bf954e1493a0f580c9edab4f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.212219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b823b27bf954e1493a0f580c9edab4f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:21.214235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:21.268759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b406decd8664d988dea0473486723c0', 'request_id': 'req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.278842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b406decd8664d988dea0473486723c0', 'request_id': 'req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.292206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b406decd8664d988dea0473486723c0', 'request_id': 'req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.446232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b823b27bf954e1493a0f580c9edab4f', 'request_id': 'req-ba433df0-5f02-45aa-9a7d-edc34980e121'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.454239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b823b27bf954e1493a0f580c9edab4f', 'request_id': 'req-ba433df0-5f02-45aa-9a7d-edc34980e121'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.464697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] The token cache is being invalidate because project 2b406decd8664d988dea0473486723c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:21.465009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.466059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.468395 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 691/1365] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:20 2022] DELETE /identity/v3/projects/2b406decd8664d988dea0473486723c0 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:21.469784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:21.475045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:21.603840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b823b27bf954e1493a0f580c9edab4f', 'request_id': 'req-ba433df0-5f02-45aa-9a7d-edc34980e121'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.640175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:21.660114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] The token cache is being invalidate because project 9b823b27bf954e1493a0f580c9edab4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:21.674561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.675688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ba433df0-5f02-45aa-9a7d-edc34980e121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:21.677783 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 675/1366] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:21 2022] DELETE /identity/v3/projects/9b823b27bf954e1493a0f580c9edab4f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:21.682702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17680f-041f-41fb-846b-ac754d61b190 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.682702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17680f-041f-41fb-846b-ac754d61b190 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.682702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17680f-041f-41fb-846b-ac754d61b190 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.682702 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 676/1367] 10.222.0.26 () {54 vars in 912 bytes} [Sun Oct 30 01:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:21.684503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f348-85bb-4a06-8e33-4edee9486d96 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.685355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f348-85bb-4a06-8e33-4edee9486d96 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.686632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f348-85bb-4a06-8e33-4edee9486d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.719351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0f4e-70e2-4b93-af73-09fb6a5a912c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:21.722438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1445f348-85bb-4a06-8e33-4edee9486d96 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:21.726287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1445f348-85bb-4a06-8e33-4edee9486d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:21.824485 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:21.889293 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 677/1368] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:21.890808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ebc-90da-4c25-a91a-4d26ce2d3d7c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.891490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ebc-90da-4c25-a91a-4d26ce2d3d7c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.891935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ebc-90da-4c25-a91a-4d26ce2d3d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.893686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:21.894454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.894747 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.895022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] PATH_INFO: `/v3/projects/8519baec3bd94590b0d88c46e6749c91` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:21.896497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8519baec3bd94590b0d88c46e6749c91)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:21.899120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:21.918739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c2d02ebc-90da-4c25-a91a-4d26ce2d3d7c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:21.920429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c2d02ebc-90da-4c25-a91a-4d26ce2d3d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:21.931782 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 678/1369] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:21.938332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59dd4-8517-4be4-b0af-de528cd35cef None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:21.939062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59dd4-8517-4be4-b0af-de528cd35cef None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:21.939453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59dd4-8517-4be4-b0af-de528cd35cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.014264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-30b59dd4-8517-4be4-b0af-de528cd35cef None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:22.015853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-30b59dd4-8517-4be4-b0af-de528cd35cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.079135 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8519baec3bd94590b0d88c46e6749c91', 'request_id': 'req-5ead053b-d042-48d4-9939-e9c7ce375463'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.087853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8519baec3bd94590b0d88c46e6749c91', 'request_id': 'req-5ead053b-d042-48d4-9939-e9c7ce375463'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.099803 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8519baec3bd94590b0d88c46e6749c91', 'request_id': 'req-5ead053b-d042-48d4-9939-e9c7ce375463'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.205181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] The token cache is being invalidate because project 8519baec3bd94590b0d88c46e6749c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:22.206532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.208331 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 679/1370] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:09:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:22.209907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5ead053b-d042-48d4-9939-e9c7ce375463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.210491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.212784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.212784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.213691 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 692/1371] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:21 2022] DELETE /identity/v3/projects/8519baec3bd94590b0d88c46e6749c91 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:22.220798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cc0c9-891f-4f16-92dd-e8ba3b0fec5c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.223323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cc0c9-891f-4f16-92dd-e8ba3b0fec5c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.224415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cc0c9-891f-4f16-92dd-e8ba3b0fec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.264476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:22.265952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.267360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a70cc0c9-891f-4f16-92dd-e8ba3b0fec5c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:22.268681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a70cc0c9-891f-4f16-92dd-e8ba3b0fec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.373540 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 680/1372] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:22.374684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:22.377859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.396894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bce92d-4f1d-4747-b745-26b9f8032dde None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:22.404006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:22.404703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.405318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.405546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] PATH_INFO: `/v3/projects/5bc4fa42794443cc92ffc01b0a27fcdd` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.407396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc4fa42794443cc92ffc01b0a27fcdd)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:22.408917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:22.431381 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 693/1373] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:22.436655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.437107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.437541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.478284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] MFA Rules not processed for user `2cfba439a18a4c8e8fa0021380128447`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:22.479883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.569169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc4fa42794443cc92ffc01b0a27fcdd', 'request_id': 'req-16894206-3d63-456b-bbe1-9e50f64a3613'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.578940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc4fa42794443cc92ffc01b0a27fcdd', 'request_id': 'req-16894206-3d63-456b-bbe1-9e50f64a3613'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.616987 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 694/1374] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:22.616987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:22.616987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.633675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07e6c-dd13-4e24-81a1-f76106bf89df None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:22.670569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc4fa42794443cc92ffc01b0a27fcdd', 'request_id': 'req-16894206-3d63-456b-bbe1-9e50f64a3613'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.691189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] The token cache is being invalidate because project 5bc4fa42794443cc92ffc01b0a27fcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:22.691729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.692821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:22.695602 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 681/1375] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:22 2022] DELETE /identity/v3/projects/5bc4fa42794443cc92ffc01b0a27fcdd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:22.696849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:22.700763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.751729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:22.752489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.753236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.753236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] PATH_INFO: `/v3/users/a10f9523b92a4043aff821768a431c6d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.759710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10f9523b92a4043aff821768a431c6d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:22.762251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:22.801577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894206-3d63-456b-bbe1-9e50f64a3613 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:22.815673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:22.816797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.816797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.817470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.819816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.832364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:22.834012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:22.845229 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 682/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:22.846512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:22.848295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.866238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac72f-4c8e-44d2-b9d8-c3ec5cb81d51 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:22.874353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:22.876689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.877296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.877773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.878953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:22.879565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:22.882291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:22.905499 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 683/1377] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:22.906711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:22.913127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:22.937587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a6de-7e42-48cf-b35f-b17d886306c1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:22.944878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:22.945925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.947629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.948121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:22.949481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:22.949818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:22.955005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17ffa-404f-40f5-9b66-d61be9c17b43 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:22.981293 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 684/1378] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:22.982421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c35de-1462-48ce-bae3-311db2c19c8b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:22.982708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c35de-1462-48ce-bae3-311db2c19c8b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:22.987787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c35de-1462-48ce-bae3-311db2c19c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.017827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10f9523b92a4043aff821768a431c6d'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:23.019069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10f9523b92a4043aff821768a431c6d'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:23.046922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10f9523b92a4043aff821768a431c6d'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:23.052357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1f5c35de-1462-48ce-bae3-311db2c19c8b None None] MFA Rules not processed for user `3e12208f0ceb41c8bb3b69cbd62e78b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:23.054386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1f5c35de-1462-48ce-bae3-311db2c19c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.139143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10f9523b92a4043aff821768a431c6d'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:23.149200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10f9523b92a4043aff821768a431c6d'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:23.161095 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 695/1379] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:22 2022] DELETE /identity/v3/users/a10f9523b92a4043aff821768a431c6d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:23.178283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:23.178283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.231834 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 685/1380] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:23.234318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.234800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.235275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.272871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9055882-9378-40df-8e8e-883035cdf7f7 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:23.273500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:23.276528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.282270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e939304-c405-415c-b035-75c381edac99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:23.283488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939304-c405-415c-b035-75c381edac99 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.284211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939304-c405-415c-b035-75c381edac99 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.284556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939304-c405-415c-b035-75c381edac99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.293040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1e939304-c405-415c-b035-75c381edac99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.418414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e939304-c405-415c-b035-75c381edac99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:23.420310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e939304-c405-415c-b035-75c381edac99 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:23.436717 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 696/1381] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:23.437664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e939304-c405-415c-b035-75c381edac99 service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:23.439239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1e939304-c405-415c-b035-75c381edac99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.493657 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 686/1382] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:23.502106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:23.505837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.545013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe6a13-9813-4f06-a2b7-03353c992cc6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:23.551687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:23.552443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.552668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.552927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.553705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:23.555717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca3063-d9b1-4b95-a881-38c61d57444d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:23.580535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e939304-c405-415c-b035-75c381edac99 service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:23.585739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:23.586546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.586654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.586933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.588193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:23.591759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799750-9b05-4c34-b790-09c675f101ec admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:23.678953 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 687/1383] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/projects => generated 399 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:23.689933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.690153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.690376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.719912 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 697/1384] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:23.722946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:23.723263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:23.723633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:23.752754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] MFA Rules not processed for user `3e12208f0ceb41c8bb3b69cbd62e78b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:23.754091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.858060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:23.858060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:23.887572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:23.925223 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 688/1385] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:23.931527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.002243 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 698/1386] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:24.002243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.026493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c096a7-b13e-42af-8bbc-b01d3b6c0572 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.045586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.045586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.045586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.045586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.046096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:24.046096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:24.100078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:24.131322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67127c-4694-478a-9422-bb69772b46dc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.144709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.145445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.145865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.146303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.147066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:24.150515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:24.190244 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 699/1387] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:24.192336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.195318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:24.220452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b094b7-758a-4129-8b35-8aef65b2b75d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.220452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.220452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.220452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.220791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] PATH_INFO: `/v3/projects/4408c8a00c184399953640947496996a` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.220791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4408c8a00c184399953640947496996a)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:24.221754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:24.302140 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 689/1388] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:23 2022] POST /identity/v3/users => generated 300 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:24.305262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.307949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:24.428742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff67ef-3bea-47ab-b436-24038f3af730 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.460274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4408c8a00c184399953640947496996a', 'request_id': 'req-4d466902-4232-4ddc-8f25-ae192aac1868'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:24.485422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4408c8a00c184399953640947496996a', 'request_id': 'req-4d466902-4232-4ddc-8f25-ae192aac1868'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:24.507254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4408c8a00c184399953640947496996a', 'request_id': 'req-4d466902-4232-4ddc-8f25-ae192aac1868'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:24.552583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.553070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.553350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.553628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.555495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:24.568319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:24.569956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:24.581026 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 690/1389] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:24.581904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.640775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] The token cache is being invalidate because project 4408c8a00c184399953640947496996a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:24.641141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:24.644943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:24.657180 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 700/1390] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:24 2022] DELETE /identity/v3/projects/4408c8a00c184399953640947496996a => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:24.658250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:24.670521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:24.790099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237367b8-e5aa-4b89-8a38-aad1838a39f8 service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.810789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.815040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.815572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.816109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.821058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:24.954121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d466902-4232-4ddc-8f25-ae192aac1868 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:24.967979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:24.973694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:24.974187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:24.974482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:24.979324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:24.981044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.013773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51ee7-6a67-4509-acc7-2bbc3a8a330a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.054903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.064280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.084700 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 701/1391] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:25.092780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.093332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.096270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.106723 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 691/1392] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:25.107618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.118325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:25.153093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] MFA Rules not processed for user `a4a4230574c846a9b4f7edd240317ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:25.154397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:25.156795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2e71-809f-4e45-a5fc-17e5cb67deaf service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.171166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.173540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.174455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.174849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.176358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.178291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.215282 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 692/1393] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:25.215282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.215282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:25.248739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0286c-d29f-4a60-ade5-032c4bf9f2fc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.263520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.264635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.264635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.264820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.265396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.270454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e9610-bb7a-410a-ad78-b9f9bbd2e8a3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.288812 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 693/1394] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:25.291090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.291360 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 702/1395] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:25.292263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.292944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.293188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.316219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21da41-6680-4837-945a-d4dee0311168 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.322416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.323170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.324486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.324782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.325636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.329965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.525046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:25.526313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:25.555493 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 703/1396] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:25 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:25.565180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.693924 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 694/1397] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:25.695179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.728494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d1eb8-f579-401a-9b81-cf89de10b3dd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.741421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.741421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.741421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.741421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.741904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.748150 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.775293 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 695/1398] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:25.776187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.791937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51daa3-dc0f-45d5-b0b8-d77743af4bce admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.797571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.799069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.799576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.799969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.800720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.804554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.828871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3862c8-6d8c-4966-a083-2d7671fcbd04 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.834613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.835635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.836125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.836922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.838012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.842698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3a57-6890-42ea-bfe2-bed0358f051c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:25.873182 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 704/1399] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:25.875488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbba746-254f-453b-83f6-7a03c81ba4d7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.876041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbba746-254f-453b-83f6-7a03c81ba4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.876472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbba746-254f-453b-83f6-7a03c81ba4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.905174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2dbba746-254f-453b-83f6-7a03c81ba4d7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:25.907624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2dbba746-254f-453b-83f6-7a03c81ba4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:25.920224 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 696/1400] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:25 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:25.921991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.941743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e4c1-cc7c-446c-b284-735fc80e56b2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:25.951399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:25.952176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.952514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.952791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] PATH_INFO: `/v3/projects/9ce2d4dd2708409a860ef36cf7d6e396/users/caf6f1235f11428f9aac7f01a4e98f98/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.962732 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 705/1401] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:25.964906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c7a-09c5-4191-b555-074e5d1dc7ff None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:25.965073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c7a-09c5-4191-b555-074e5d1dc7ff None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:25.965484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c7a-09c5-4191-b555-074e5d1dc7ff None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:25.968319 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 706/1402] 10.222.0.26 () {54 vars in 912 bytes} [Sun Oct 30 01:09:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:25.971960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05c7a-09c5-4191-b555-074e5d1dc7ff None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:25.981725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce2d4dd2708409a860ef36cf7d6e396, user_id=caf6f1235f11428f9aac7f01a4e98f98, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:25.988009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.000896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05c7a-09c5-4191-b555-074e5d1dc7ff None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.011919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.012505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.017113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.017113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] PATH_INFO: `/v3/projects/6d67aab66bdc4e4884a738899629b3b6/users/291ccfc5f76a4924a1f6c983f92eacab/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.073194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67aab66bdc4e4884a738899629b3b6, user_id=291ccfc5f76a4924a1f6c983f92eacab, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.092909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.148464 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 697/1403] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:25 2022] PUT /identity/v3/projects/9ce2d4dd2708409a860ef36cf7d6e396/users/caf6f1235f11428f9aac7f01a4e98f98/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:26.150563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.173754 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 707/1404] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:25 2022] PUT /identity/v3/projects/6d67aab66bdc4e4884a738899629b3b6/users/291ccfc5f76a4924a1f6c983f92eacab/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:26.177792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.343796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70cfa8-bd1a-4720-abf9-02186e0459e8 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.362221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.362947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.363300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.364576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.365411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:26.365791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.367721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ed951-16c8-4975-8a87-22778c301b60 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.383785 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 708/1405] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:26.385889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161e97-f4a0-4004-8583-0d20ec6cb8ca None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.386247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161e97-f4a0-4004-8583-0d20ec6cb8ca None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.386596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161e97-f4a0-4004-8583-0d20ec6cb8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.398651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c2ee-0ded-429b-9dca-4c6a02a95782 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.409751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-07161e97-f4a0-4004-8583-0d20ec6cb8ca None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:26.411114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-07161e97-f4a0-4004-8583-0d20ec6cb8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:26.413889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.418275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.418275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.418275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.418275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.418275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abf8a0-36f9-4caa-82a1-cc462b5e4678 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.463271 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 698/1406] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:26.464341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.464656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.464927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.499837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] MFA Rules not processed for user `caf6f1235f11428f9aac7f01a4e98f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:26.504476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:26.620929 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 709/1407] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:26.622405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.622698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.622975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.686547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] MFA Rules not processed for user `291ccfc5f76a4924a1f6c983f92eacab`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:26.688768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:26.699455 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 699/1408] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:26.705629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.728110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f0591-27ca-4285-ac5b-0f4e87905252 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.747118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.747794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.748085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.748369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.749367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.751675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.800984 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 710/1409] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:26.808858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.814363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9890c-76e9-4f32-9433-6e2631c205aa None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.820556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.821370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.821664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.821941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.822618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.824482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.836296 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 711/1410] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:26.843290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.843290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:26.858279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4792e9-3beb-4540-8c8c-7507fa95963f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.866898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:26.867969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:26.869209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:26.869691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:26.889652 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 700/1411] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:26.892491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.896246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:26.900336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:26.917790 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 712/1412] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:26.921147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:26.935199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9896d6-915e-4ff1-b623-e3502656140d service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:26.980112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:27.047250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392f999-1784-49b1-b7c1-3d1ef161d782 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.053275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.053761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.053975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.054197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.054554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.055219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.056712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.057340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.057819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.058706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.061441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.063919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.194165 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 713/1414] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/projects => generated 367 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:27.194975 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 701/1414] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:26 2022] POST /identity/v3/projects => generated 399 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:27.195060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:27.195133 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:27.236749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:27.448551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b639c1-961e-429e-8051-91b4e69f295c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.474534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.475930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.477446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.478353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.480863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.484611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.527906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac86b-ac51-406d-9b3b-15ff06f5be69 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.534454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.535297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.535689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.536040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] PATH_INFO: `/v3/projects/34a9569845304afd9dbe1047b87fa17b/users/fccb934331024c6991c4828e1c0eb531/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.571082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9569845304afd9dbe1047b87fa17b, user_id=fccb934331024c6991c4828e1c0eb531, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.583359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.645443 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 702/1415] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:27 2022] POST /identity/v3/users => generated 289 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:27.645443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:27.659263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258f65-4c4a-4a00-812c-c25db22522e8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.689524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.690439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.691980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.694602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.705011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.706271 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 714/1416] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:27 2022] PUT /identity/v3/projects/34a9569845304afd9dbe1047b87fa17b/users/fccb934331024c6991c4828e1c0eb531/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:27.711533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.720719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:27.871911 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 703/1417] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:27 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:27.872966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:27.909761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e02fee-4ba1-4e4f-97d5-9fefa3cc320c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.910726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c952f-02e0-4735-b7af-bfb89d7ebccc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:27.918905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.920190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.920720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.922465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.923783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.925079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:27.925970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.926252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.926524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:27.927260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:27.927789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.939914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73035dfc-81b0-4cab-bb23-2e39f3f4d427 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:27.967869 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 704/1418] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:27.970370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:27.970836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:27.971262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.010810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] MFA Rules not processed for user `fccb934331024c6991c4828e1c0eb531`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:28.012590 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:28.076519 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 715/1419] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:27 2022] POST /identity/v3/users => generated 306 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:28.077037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.123160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e29442-6417-4554-af48-74b59d130ab0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.138566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.140742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.142786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.143531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.144730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.150116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.193588 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 716/1420] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:28.196617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.270275 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 705/1421] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:28.275633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bca8aa-383d-4664-abee-2d6c2049b1b3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.288151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.288306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.288306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.288306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.288306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.288583 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.289545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.312295 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 717/1422] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:28.313639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.356846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17eaae-7aff-49af-846e-dd6c30dc7399 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.359649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e072f24-d5a2-47ca-a92b-e4fc82982e14 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.385641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.386647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.386939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.387129 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.387596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.387919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.388293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.388689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] PATH_INFO: `/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b/users/bb63722318b14ef08c78595a2dff515c/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.388796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.408482 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.447774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76705e3b237142e18ba5bdb2cbb10f6b, user_id=bb63722318b14ef08c78595a2dff515c, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.450508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.454197 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 706/1423] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:28.455656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.529993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87026a17-c0ad-407f-9ff7-fa30adecd845 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.584810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.584810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.584810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.584810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.585531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.585531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.590616 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 718/1424] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:28 2022] PUT /identity/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b/users/bb63722318b14ef08c78595a2dff515c/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:28.591799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.731544 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 707/1425] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:28 2022] POST /identity/v3/projects => generated 403 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:28.732536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:28.863900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631fc60-2db4-4c20-901d-87b253ba57f9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.923460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910564b-aa0e-4f68-858d-161fa91a424c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:28.932305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.933250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.933537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.934210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] PATH_INFO: `/v3/projects/d1c9f837797e4695979e53b7e70336d2/users/9be88306987c48fa88684d9ac9441959/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.956720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c9f837797e4695979e53b7e70336d2, user_id=9be88306987c48fa88684d9ac9441959, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.961311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:28.964821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:28.965642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:28.965923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:28.966197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] PATH_INFO: `/v3/projects/4478faebdde8453a9ef1ac61018a9fdc/users/8edd949e15504575ae1a1a7ca3338b66/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:28.987137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4478faebdde8453a9ef1ac61018a9fdc, user_id=8edd949e15504575ae1a1a7ca3338b66, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:28.989931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.065971 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 708/1426] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:28 2022] PUT /identity/v3/projects/d1c9f837797e4695979e53b7e70336d2/users/9be88306987c48fa88684d9ac9441959/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:29.070628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.096115 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 719/1427] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:28 2022] PUT /identity/v3/projects/4478faebdde8453a9ef1ac61018a9fdc/users/8edd949e15504575ae1a1a7ca3338b66/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:29.103706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.166262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5af9-288b-429e-8cac-cbdcc0b9a2f3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.182634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:29.193176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e421d-d7ff-4551-bfac-94645f7b2ef2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.199300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.200581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.201229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.201229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.201768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.203888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b1568-de64-423c-9eb9-906be09e1ddf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.249336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.249959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.250498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.250768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.251416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.253051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.263507 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 709/1428] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:29.264370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.273842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f706b-9e2b-4086-bbb8-eb5b406eb1d0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.282934 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.283911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.284235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.284513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.285209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.286910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b5e6-df4e-4553-9da4-b42a33746ef4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.297999 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 710/1429] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:29.300158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.300548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.301145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.332396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] MFA Rules not processed for user `8edd949e15504575ae1a1a7ca3338b66`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:29.333724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:29.385709 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 720/1430] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:29 2022] POST /identity/v3/users => generated 308 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:29.387078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.387370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.387671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.413624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] MFA Rules not processed for user `2cfba439a18a4c8e8fa0021380128447`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:29.414984 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 711/1431] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:29.415691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.416667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:29.425840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:29.432569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056294dd-3401-410d-89a8-9dbe49d132ae None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.437848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.438534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.438820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.439100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] PATH_INFO: `/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b/users/bb63722318b14ef08c78595a2dff515c/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.441091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76705e3b237142e18ba5bdb2cbb10f6b, user_id=bb63722318b14ef08c78595a2dff515c, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.443816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.455565 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 712/1432] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:29 2022] PUT /identity/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b/users/bb63722318b14ef08c78595a2dff515c/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:29.461800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.489378 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 721/1433] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:29.490371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.508606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a79a-e6d9-4871-8347-6b568517ba9b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.566540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b307b3e-9faa-4cab-8d44-80c56f099a79 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.573841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.574775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.575446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.575977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] PATH_INFO: `/v3/projects/d1c9f837797e4695979e53b7e70336d2/users/9be88306987c48fa88684d9ac9441959/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.579276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c9f837797e4695979e53b7e70336d2, user_id=9be88306987c48fa88684d9ac9441959, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.583498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.584365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.584716 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.585055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.585929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.588785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.606301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be66492-cdd8-41c9-96ec-fd205122c96c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:29.613572 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 722/1434] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:29.614540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.621876 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 713/1435] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:29 2022] PUT /identity/v3/projects/d1c9f837797e4695979e53b7e70336d2/users/9be88306987c48fa88684d9ac9441959/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:29.625969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.628018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.629468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.637052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72936e5-0340-490b-8414-a2d9378c41b1 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:29.719671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] MFA Rules not processed for user `bb63722318b14ef08c78595a2dff515c`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:29.726493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:29.866346 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 714/1436] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:29.867883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:29.898477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:29.903527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:29.903962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:29.904249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:29.908128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:29.910193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b75e-ecfe-4d85-88e3-239a83960181 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.048627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e24d7-63c3-4b37-9bc6-1a565ac4bb40 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.069384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.070091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.070304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.070644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] PATH_INFO: `/v3/projects/4d9b972ba669409c8612a94b6ba437e9/users/9990c29b0a81440c9c0493b7affb9c81/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.093971 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 723/1437] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:29 2022] POST /identity/v3/projects => generated 377 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:30.093971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.093971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.093971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.120473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9b972ba669409c8612a94b6ba437e9, user_id=9990c29b0a81440c9c0493b7affb9c81, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:30.130527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.204527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] MFA Rules not processed for user `9be88306987c48fa88684d9ac9441959`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:30.206670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:30.235367 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 715/1438] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:29 2022] PUT /identity/v3/projects/4d9b972ba669409c8612a94b6ba437e9/users/9990c29b0a81440c9c0493b7affb9c81/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:30.235367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:30.235546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:30.360164 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 724/1439] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:30 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:30.366272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:30.377069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf61a-0f2e-4390-87e2-677ffcb8ff95 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.393639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.399137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.399747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.400055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.536269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:30.538635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.539848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3c9f1-86b4-478b-b0ae-8ac7bd1c2239 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.553799 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 716/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:30.555157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:30.741873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.742934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.743318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.744334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.745091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:30.748387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.769690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58559b61-5f50-43b3-b292-ed6b54bbac71 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.791363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.791363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.791363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.791363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.791841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:30.791841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.813935 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 717/1441] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:30.815102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:30.819752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:30.848016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c487d-16b1-47be-acb0-487be3a5314a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.853817 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.857162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.857690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.858173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.944829 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 725/1442] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:30 2022] POST /identity/v3/users => generated 295 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:30.945860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:30.947824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:30.964317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ef989-cc6c-4a56-bf06-b47aab992516 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:30.973193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:30.979695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:30.979695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:30.979695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] PATH_INFO: `/v3/projects/4d9b972ba669409c8612a94b6ba437e9/users/9990c29b0a81440c9c0493b7affb9c81/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:30.980286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9b972ba669409c8612a94b6ba437e9, user_id=9990c29b0a81440c9c0493b7affb9c81, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:30.983495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d91393-e852-42ca-b531-28640caa9b3a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:30.999686 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 726/1443] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:30 2022] PUT /identity/v3/projects/4d9b972ba669409c8612a94b6ba437e9/users/9990c29b0a81440c9c0493b7affb9c81/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:31.001851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.002344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.002715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.025715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:31.025715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:31.057912 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 718/1444] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:31.062457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.104733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd5ad1-7b3f-45b7-a5ee-5a311f425720 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:31.214888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:31.217309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:31.280169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:31.280823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.281099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.281382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.282045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:31.283706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815bca9-6986-4699-8a3e-2a6f5640666d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:31.318001 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 719/1445] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:31.319928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.320232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.320736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.390055 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 727/1446] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:09:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:31.390752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.393203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] MFA Rules not processed for user `9990c29b0a81440c9c0493b7affb9c81`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:31.407136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:31.514672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc9202-4043-4bd0-91e5-40592d832da9 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:31.519023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:31.523222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:31.524405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.524632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.524909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.568316 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 720/1447] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:31.568316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.592831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e0fcd-1d23-4f1d-8b70-29ec555ababd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:31.608972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:31.609666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.609946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.610225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] PATH_INFO: `/v3/projects/00de37c042ab410880d497bc938bc040/users/eca2326a391e458283b88ee9ec4cf374/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.633568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:31.635628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:31.644008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00de37c042ab410880d497bc938bc040, user_id=eca2326a391e458283b88ee9ec4cf374, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:31.648589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:31.658281 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 728/1448] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:31.659723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.670755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397295f-e863-4c0c-97aa-e9e975c9fb70 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:31.690700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:31.691559 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:31.691739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:31.692394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:31.768245 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 721/1449] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:31 2022] PUT /identity/v3/projects/00de37c042ab410880d497bc938bc040/users/eca2326a391e458283b88ee9ec4cf374/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:31.772447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.859036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:31.861297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:31.905156 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 729/1450] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:31.905156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:31.907984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:31.992079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e114-d9b4-4270-b719-066374cbf269 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:32.012323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:32.013208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.013492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.013775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.014474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:32.020015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:32.060841 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 722/1451] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:32.065895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:32.083925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409043-c7d8-415e-a957-4fa27f2b98e2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:32.091254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:32.092613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.092909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.093190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] PATH_INFO: `/v3/projects/00de37c042ab410880d497bc938bc040/users/eca2326a391e458283b88ee9ec4cf374/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.099104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00de37c042ab410880d497bc938bc040, user_id=eca2326a391e458283b88ee9ec4cf374, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:32.111286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630891c6-c0e8-4ea0-819e-9c8ee1d0af77 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:32.159132 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 723/1452] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:32 2022] PUT /identity/v3/projects/00de37c042ab410880d497bc938bc040/users/eca2326a391e458283b88ee9ec4cf374/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:32.174413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.179844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.179844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.200507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:32.265515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] MFA Rules not processed for user `eca2326a391e458283b88ee9ec4cf374`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:32.267777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:32.424983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a6fd-3bb9-4f51-8dd0-b1a64877b80d service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:32.452247 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 724/1453] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:32.493156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:32.567207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:32.607042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:32.608566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.608566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.608566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.701642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72591752-2b75-49c5-a0a4-20b109a494ae None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:32.709913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:32.714289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.714892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.715006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.785133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:32.785531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2e8be-fe19-494f-aa40-0af4df3eab36 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:32.800196 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 730/1454] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 898 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:09:32.809319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:32.809319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:32.809319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:32.830636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:32.832162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:32.911286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:32.913777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:32.948298 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 725/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:32.949331 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 731/1456] 10.222.0.61 () {58 vars in 1015 bytes} [Sun Oct 30 01:09:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:32.976418 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:32.978806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:33.009514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebad5-c993-44eb-86b5-b522b4bbdc83 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:33.029931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:33.029931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:33.029931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:33.029931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:33.035435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:33.049015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:33.049604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8811c2a-9e51-4559-b128-8f15390830b7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:33.050476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:33.056657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:33.057951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:33.058275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:33.058670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:33.100892 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 726/1457] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:33.240067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:33.246513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09ab37-61bb-490d-9df9-1102e55ba1ba service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:33.272467 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 732/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:33.501410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:33.880752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:34.030265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923175c0-c4ca-4f33-82ca-4fe3b2c8daf8 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:34.038973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:34.040577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:34.040577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:34.040802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] PATH_INFO: `/v3/users/fccb934331024c6991c4828e1c0eb531` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:34.042417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccb934331024c6991c4828e1c0eb531)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:34.044768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:34.562491 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:34.799192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccb934331024c6991c4828e1c0eb531'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:34.799994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccb934331024c6991c4828e1c0eb531'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:34.815407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccb934331024c6991c4828e1c0eb531'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:34.816146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccb934331024c6991c4828e1c0eb531'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:34.849477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccb934331024c6991c4828e1c0eb531'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:34.944720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:34.971703 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 727/1459] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:33 2022] DELETE /identity/v3/users/fccb934331024c6991c4828e1c0eb531 => generated 0 bytes in 1092 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:34.976976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:34.978133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:34.978554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:35.051820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:35.053069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:35.348061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 733/1460] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:35.350272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:35.350272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:35.350272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8b1d-a944-491e-a2f4-f50b76dbe84e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:35.370942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:35.371632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:35.371904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:35.372180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:35.385422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:35.387216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd2876-fbc3-4c83-a14e-498cf1b00a99 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:35.403019 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 728/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:35.525774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:35.530193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:35.553360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900952-49d9-4b14-882c-5ee6c30fba66 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:35.564069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:35.569007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:35.570561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:35.571821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] PATH_INFO: `/v3/users/0d769cecaee04e2ab9b9fc22d53003ae` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:35.575160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d769cecaee04e2ab9b9fc22d53003ae)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:35.579517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:35.992200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d769cecaee04e2ab9b9fc22d53003ae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:35.995194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d769cecaee04e2ab9b9fc22d53003ae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:36.065762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d769cecaee04e2ab9b9fc22d53003ae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:36.175507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d769cecaee04e2ab9b9fc22d53003ae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:36.206233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d769cecaee04e2ab9b9fc22d53003ae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:36.236213 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 734/1462] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:35 2022] DELETE /identity/v3/users/0d769cecaee04e2ab9b9fc22d53003ae => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:36.244881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:36.244881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:36.245537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:36.377099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:36.377099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:36.378843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:36.378843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:36.597007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02db5-ca6f-4b9e-abf5-d67b114a7da3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:36.603224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:36.604267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:36.604778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:36.605543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] PATH_INFO: `/v3/projects/34a9569845304afd9dbe1047b87fa17b` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:36.607268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a9569845304afd9dbe1047b87fa17b)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:36.609326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:36.645151 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 729/1463] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:36.655971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:36.658110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:36.892668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:36.894635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4fa82-30a7-4e45-81df-7de7f4316f78 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:36.894635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:36.894635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:36.894635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97371ca0-7c99-42bd-a57f-aa8b42c4444c service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 730/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872906b3-33be-403b-9a17-48f6cc53975c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:36.894975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872906b3-33be-403b-9a17-48f6cc53975c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872906b3-33be-403b-9a17-48f6cc53975c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 731/1465] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ff74-d863-4718-b453-a6b981049774 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ff74-d863-4718-b453-a6b981049774 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ff74-d863-4718-b453-a6b981049774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:36.895419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8144ff74-d863-4718-b453-a6b981049774 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:36.895888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8144ff74-d863-4718-b453-a6b981049774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:36.954963 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 732/1466] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:09:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:36.995060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ff74-d863-4718-b453-a6b981049774 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:36.996771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8144ff74-d863-4718-b453-a6b981049774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:37.026390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ff74-d863-4718-b453-a6b981049774 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:37.033124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:37.033755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:37.034041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:37.034332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:37.048755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:37.050578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:37.078859 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 733/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:37.134167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a9569845304afd9dbe1047b87fa17b', 'request_id': 'req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:37.148146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a9569845304afd9dbe1047b87fa17b', 'request_id': 'req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:37.157260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a9569845304afd9dbe1047b87fa17b', 'request_id': 'req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:37.226840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] The token cache is being invalidate because project 34a9569845304afd9dbe1047b87fa17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:37.227248 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:37.228786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:37.234979 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 735/1468] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:36 2022] DELETE /identity/v3/projects/34a9569845304afd9dbe1047b87fa17b => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:37.238908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:37.266809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d2cc-a9a4-4732-b930-6c18ea2fb176 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:37.394193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:37.397130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:37.459461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:37.460664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:37.461183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:37.461727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] PATH_INFO: `/v3/users/9990c29b0a81440c9c0493b7affb9c81` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:37.463418 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9990c29b0a81440c9c0493b7affb9c81)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:37.465789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:37.560666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f70f5-5946-42b1-acd0-60ebdbbc3cf4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:37.564932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:37.565810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:37.566429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:37.566737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:37.569028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:37.696909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:37.698395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:37.715054 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 736/1469] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:37.827987 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:38.095447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9990c29b0a81440c9c0493b7affb9c81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.100927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9990c29b0a81440c9c0493b7affb9c81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.137197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9990c29b0a81440c9c0493b7affb9c81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.157218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9990c29b0a81440c9c0493b7affb9c81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.200635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9990c29b0a81440c9c0493b7affb9c81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.306315 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 734/1470] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:37 2022] DELETE /identity/v3/users/9990c29b0a81440c9c0493b7affb9c81 => generated 0 bytes in 1068 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:38.553772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:38.561253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:38.615277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0d14d-a418-4dbf-b915-09ad979d4ba6 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:38.722138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:38.723969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:38.724804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:38.725463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] PATH_INFO: `/v3/projects/42a24c56418f48d48ea02785cfced7f4` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:38.728132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a24c56418f48d48ea02785cfced7f4)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:38.730427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:38.912026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a24c56418f48d48ea02785cfced7f4', 'request_id': 'req-f90e4779-fcc0-4cc4-8a83-08a119e01802'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.929895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a24c56418f48d48ea02785cfced7f4', 'request_id': 'req-f90e4779-fcc0-4cc4-8a83-08a119e01802'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:38.940575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a24c56418f48d48ea02785cfced7f4', 'request_id': 'req-f90e4779-fcc0-4cc4-8a83-08a119e01802'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.054223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] The token cache is being invalidate because project 42a24c56418f48d48ea02785cfced7f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:39.054581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.056595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f90e4779-fcc0-4cc4-8a83-08a119e01802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.061577 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 737/1471] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:38 2022] DELETE /identity/v3/projects/42a24c56418f48d48ea02785cfced7f4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:39.062765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:39.064359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:39.085378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd156-dca7-4a79-bff0-75cca2747deb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:39.254092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:39.254901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:39.256087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:39.257075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] PATH_INFO: `/v3/users/4600b31dd5b74f01ba6319787411975d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:39.262236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4600b31dd5b74f01ba6319787411975d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:39.268720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:39.838132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4600b31dd5b74f01ba6319787411975d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.839024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4600b31dd5b74f01ba6319787411975d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.862528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4600b31dd5b74f01ba6319787411975d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.862528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4600b31dd5b74f01ba6319787411975d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:39.881777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:39.914341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4600b31dd5b74f01ba6319787411975d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f68a3-5893-43de-b44b-f3118c06aed8 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f68a3-5893-43de-b44b-f3118c06aed8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f68a3-5893-43de-b44b-f3118c06aed8 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 738/1472] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:40.036464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:40.037585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:40.075260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:40.076854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:40.216137 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 739/1473] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:09:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:40.251009 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 735/1474] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:39 2022] DELETE /identity/v3/users/4600b31dd5b74f01ba6319787411975d => generated 0 bytes in 1189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:40.322237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:40.328037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:40.386684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d626d1b-d4ec-4524-809a-4663d0b6cea5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:40.547062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:40.547757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:40.548039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:40.548320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:40.564408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:40.566706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:40.591856 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 740/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:40.860694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:40.863115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:40.936642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:41.248792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f163b-7f18-489b-9eeb-0c7788ec51e0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:41.248792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:41.248792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:41.248792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:41.250890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] PATH_INFO: `/v3/projects/4d9b972ba669409c8612a94b6ba437e9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:41.250890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9b972ba669409c8612a94b6ba437e9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:41.250890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:41.489915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:41.492619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:41.511822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9b972ba669409c8612a94b6ba437e9', 'request_id': 'req-4964bb08-8bc4-42de-92af-fb3a92fc1a97'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:41.531503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9b972ba669409c8612a94b6ba437e9', 'request_id': 'req-4964bb08-8bc4-42de-92af-fb3a92fc1a97'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:41.552991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69a894-71ba-47f4-8edd-bd16a9f4c03a service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:41.625069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9b972ba669409c8612a94b6ba437e9', 'request_id': 'req-4964bb08-8bc4-42de-92af-fb3a92fc1a97'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:41.660499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] The token cache is being invalidate because project 4d9b972ba669409c8612a94b6ba437e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:41.668174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:41.675740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4964bb08-8bc4-42de-92af-fb3a92fc1a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:41.684998 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 736/1476] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:40 2022] DELETE /identity/v3/projects/4d9b972ba669409c8612a94b6ba437e9 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:41.701722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f9a31-2260-4d2e-8185-77dad51e31eb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:41.701943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f9a31-2260-4d2e-8185-77dad51e31eb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:41.702169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f9a31-2260-4d2e-8185-77dad51e31eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:41.757147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d78f9a31-2260-4d2e-8185-77dad51e31eb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:41.758940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d78f9a31-2260-4d2e-8185-77dad51e31eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:41.784630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:41.786679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:41.788358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:41.788873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] PATH_INFO: `/v3/projects/ba2aebddaf564657919cc25bd8b7b506` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:41.791543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2aebddaf564657919cc25bd8b7b506)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:41.796514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:41.803335 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 737/1477] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:41.805694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:41.808312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:41.808312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:41.917156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:42.075746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:42.075746 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 738/1478] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:42.075746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:42.075746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:42.075746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37931600-ff1b-4321-b9d8-42459dd1f418 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:42.076857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:42.076857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:42.076857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:42.076857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] PATH_INFO: `/v3/users/caf6f1235f11428f9aac7f01a4e98f98` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:42.077804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf6f1235f11428f9aac7f01a4e98f98)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:42.077804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:42.333000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2aebddaf564657919cc25bd8b7b506', 'request_id': 'req-90a5033b-587e-4b8f-af59-b2fd571d14cd'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.344775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2aebddaf564657919cc25bd8b7b506', 'request_id': 'req-90a5033b-587e-4b8f-af59-b2fd571d14cd'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.366770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2aebddaf564657919cc25bd8b7b506', 'request_id': 'req-90a5033b-587e-4b8f-af59-b2fd571d14cd'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.500482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] The token cache is being invalidate because project ba2aebddaf564657919cc25bd8b7b506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:42.502977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.504309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.507328 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 741/1479] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:41 2022] DELETE /identity/v3/projects/ba2aebddaf564657919cc25bd8b7b506 => generated 0 bytes in 1019 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:42.509676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:42.512994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:42.516851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf6f1235f11428f9aac7f01a4e98f98'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.517718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf6f1235f11428f9aac7f01a4e98f98'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.536277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf6f1235f11428f9aac7f01a4e98f98'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.537117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf6f1235f11428f9aac7f01a4e98f98'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.562229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-895f5603-627a-4c11-a912-37176ce0fcbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf6f1235f11428f9aac7f01a4e98f98'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:42.708681 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 739/1480] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:41 2022] DELETE /identity/v3/users/caf6f1235f11428f9aac7f01a4e98f98 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:42.715795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50850-45a5-4355-9789-0952cc07afc2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:42.715795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50850-45a5-4355-9789-0952cc07afc2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:42.715795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50850-45a5-4355-9789-0952cc07afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:42.726657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5033b-587e-4b8f-af59-b2fd571d14cd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:42.760623 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-79b50850-45a5-4355-9789-0952cc07afc2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:42.763335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-79b50850-45a5-4355-9789-0952cc07afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:42.907910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:42.925042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:42.944782 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 740/1481] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:42.944782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:42.944782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:42.944782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:42.945653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:42.945653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:42.945653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:42.945653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:42.945653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a11204-2e41-4960-ad0d-de26d3ae72e4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:42.951475 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 742/1482] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:42.953012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:42.953362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:42.953837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:42.984703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:42.984940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:42.988065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:42.989757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:43.000241 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 743/1483] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:43.001124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.003005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:43.013114 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 741/1484] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:43.014365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.026630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b216d9d-e314-4350-8e92-22ca5b512cb4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.036974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a8ec1-4ce6-44da-b8ba-d43d5f79d8c2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.043902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.044948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.045385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.045760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.046921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.047116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.047625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.047914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.048209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.048995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:43.049238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.051179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.052521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.083271 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 742/1485] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:43.088074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.111262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9075f4-104d-4ff7-9c66-b8800d73abbb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.120400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.121300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.121746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.122219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.123045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.125723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.257784 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 744/1486] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:43 2022] POST /identity/v3/projects => generated 381 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:43.260342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.381377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:43.401759 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 743/1487] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:43 2022] POST /identity/v3/projects => generated 365 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:43.407422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.425273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:43.484375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588580dd-ca97-4e80-a3b2-87b8666e6443 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.576809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26447-89a1-41ec-8ba5-f1222f259f26 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.594881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.595698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.595990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.596481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.651549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.656446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.656446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.657096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.658175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.672406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.733904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.736803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.758658 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 744/1488] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:43.760808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:43.783120 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dab254-ab17-4b5e-89b7-93dcf441a051 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:43.794714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:43.797360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:43.797360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:43.797360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:43.797360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:43.799806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:43.873937 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 745/1489] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:43 2022] POST /identity/v3/users => generated 291 bytes in 615 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:44.008140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.020313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784695-89d7-429c-977d-2fa6ecf834ab admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.027829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.029126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.029608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.030234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.031397 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.033978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.049648 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 746/1490] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:44.054084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.068968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff18bb-58e2-4a22-95ad-70b9d9c6a349 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.074680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.075511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.075969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.076446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.077235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.079185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.097420 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 747/1491] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:44.101914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.129871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad52a4-7955-465d-ab92-eca69d5048b0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.134638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.136149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.136557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.137055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] PATH_INFO: `/v3/projects/193926ed37184666859980c480f36157/users/4ebc13e4a9b4411a9b962587643abaf3/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.162909 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 745/1492] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:43 2022] POST /identity/v3/users => generated 283 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:44.169552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.190684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=193926ed37184666859980c480f36157, user_id=4ebc13e4a9b4411a9b962587643abaf3, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.203115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452af8b0-c401-4179-bb7d-e71e9951a26b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.208154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2728-0f53-4b01-89ed-7bc32df5166e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.223324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.225559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.225559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.225559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.225559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.227371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.244188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 746/1493] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:44.252870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.276239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fbdf7-0db1-44b6-b2ef-a2bd23e220e8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.283679 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 748/1494] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:44 2022] PUT /identity/v3/projects/193926ed37184666859980c480f36157/users/4ebc13e4a9b4411a9b962587643abaf3/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:44.291841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.293777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.294485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.359718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] MFA Rules not processed for user `4ebc13e4a9b4411a9b962587643abaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:44.361623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:44.365022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.365877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.366275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.367059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.368544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.370421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.393797 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 747/1495] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:44.395152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.432276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656357-76ae-43ad-988e-2d6d869325b2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.456270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.461075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.461075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.461075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] PATH_INFO: `/v3/projects/7ee155a421704255933d003ea9d593a9/users/d2d7dfdc3af24ec2ac27115540c20395/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.504272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee155a421704255933d003ea9d593a9, user_id=d2d7dfdc3af24ec2ac27115540c20395, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.508902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee4252-d164-44f8-8c43-e3978591ca10 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.528481 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 749/1496] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:44.534789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.564817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b03e7-2623-42d3-b45c-7e5c559586a5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.575357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.582768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.582768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.582768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.582768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.582768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.605834 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 748/1497] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:44 2022] PUT /identity/v3/projects/7ee155a421704255933d003ea9d593a9/users/d2d7dfdc3af24ec2ac27115540c20395/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:44.621278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.622506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.623636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.726463 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 750/1498] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:44 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:44.731190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.744931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] MFA Rules not processed for user `d2d7dfdc3af24ec2ac27115540c20395`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:44.748302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:44.891367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ee41e-b9f2-466b-937a-c7a777dd77d4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.901303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.902119 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.902892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.903746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.905564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:44.908858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:44.942745 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 749/1499] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:44.942745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:44.944303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:44.976891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d5d6-5ec5-4ae8-9e85-bff0b477a88a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:44.989893 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:44.992954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:44.992954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:44.992954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] PATH_INFO: `/v3/users/291ccfc5f76a4924a1f6c983f92eacab` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:44.994392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291ccfc5f76a4924a1f6c983f92eacab)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.001156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.095646 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 751/1500] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:44 2022] POST /identity/v3/users => generated 298 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:45.096705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.110149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217ebc-5d36-4f40-a827-064d9d6afe3b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.115266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.116150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.116363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.119279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.119279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.130850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.186075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:45.248931 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 752/1501] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:45 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:45.253670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.299830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291ccfc5f76a4924a1f6c983f92eacab'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.301374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291ccfc5f76a4924a1f6c983f92eacab'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.314997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291ccfc5f76a4924a1f6c983f92eacab'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.315970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291ccfc5f76a4924a1f6c983f92eacab'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.324902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a366e8-e310-4652-a7f7-eb1007627cbe admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.331021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.331834 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.332298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.332506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.333483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291ccfc5f76a4924a1f6c983f92eacab'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.333662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.335216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.353181 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 753/1502] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:45.355053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.357875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.371449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443cb7d-6fec-4797-a87a-bdeea671c99b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.376528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.377180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.377466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.377791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] PATH_INFO: `/v3/users/8edd949e15504575ae1a1a7ca3338b66` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.378957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8edd949e15504575ae1a1a7ca3338b66)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.380981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.417206 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 750/1503] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:44 2022] DELETE /identity/v3/users/291ccfc5f76a4924a1f6c983f92eacab => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:45.418312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.419967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.496579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4509f-3409-4f6d-a356-dd48f4ece1b6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.501829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.502476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.502756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.503030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] PATH_INFO: `/v3/projects/9ce2d4dd2708409a860ef36cf7d6e396` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.504203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce2d4dd2708409a860ef36cf7d6e396)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.506139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.575423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8edd949e15504575ae1a1a7ca3338b66'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.576153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8edd949e15504575ae1a1a7ca3338b66'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.589942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8edd949e15504575ae1a1a7ca3338b66'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.624398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce2d4dd2708409a860ef36cf7d6e396', 'request_id': 'req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.630068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce2d4dd2708409a860ef36cf7d6e396', 'request_id': 'req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.687357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8edd949e15504575ae1a1a7ca3338b66'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.697415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8edd949e15504575ae1a1a7ca3338b66'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.704431 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 754/1504] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:45 2022] DELETE /identity/v3/users/8edd949e15504575ae1a1a7ca3338b66 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:45.705789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.708120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.778145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce2d4dd2708409a860ef36cf7d6e396', 'request_id': 'req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.781083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921fea5-49aa-477b-9885-bc0994fce05b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.788241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.788241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.788241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.788241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.788549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.790057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfe8d9-d50d-4800-a7e6-c6a529253b9e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.791060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] The token cache is being invalidate because project 9ce2d4dd2708409a860ef36cf7d6e396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:45.791357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.792056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:45.793587 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 751/1505] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:45 2022] DELETE /identity/v3/projects/9ce2d4dd2708409a860ef36cf7d6e396 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:45.794331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.795693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.863351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8bc9-a564-4e74-9986-1fe3c3ae52b1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.868972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.869660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.869948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.873994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] PATH_INFO: `/v3/projects/67cdd608eb694a82b9180ef05795f672/users/21aaa4a59baf43898e8e7220da337a42/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.874111 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 755/1506] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:45 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:45.874111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0874fa6-1321-43bd-922c-0158004b74bc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.874111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0874fa6-1321-43bd-922c-0158004b74bc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.874111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0874fa6-1321-43bd-922c-0158004b74bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.889758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67cdd608eb694a82b9180ef05795f672, user_id=21aaa4a59baf43898e8e7220da337a42, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.892594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.893090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f0874fa6-1321-43bd-922c-0158004b74bc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:45.894299 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f0874fa6-1321-43bd-922c-0158004b74bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.902200 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 756/1507] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:45.903414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.903723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.904000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.922885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:45.924242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.932953 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 757/1508] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:45.933885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.935140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:45.948374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f173d-8b39-4bbe-a570-03e58612aac0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:45.953340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:45.958142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:45.958142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:45.958142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] PATH_INFO: `/v3/users/9be88306987c48fa88684d9ac9441959` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:45.958142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be88306987c48fa88684d9ac9441959)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:45.958142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:45.989717 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 752/1509] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:45 2022] PUT /identity/v3/projects/67cdd608eb694a82b9180ef05795f672/users/21aaa4a59baf43898e8e7220da337a42/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:45.993163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:45.993163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.060175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753405b-6d0e-40c2-81d7-787caee55868 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.064835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.065445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.065734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.066010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.066657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.068331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.078910 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 753/1510] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:46.079826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.084315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.141462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126d7b2-3c4c-4a62-94b5-ef894deab36f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.146082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.147441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.147914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.148041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be88306987c48fa88684d9ac9441959'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:46.148470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.148982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be88306987c48fa88684d9ac9441959'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:46.152935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.163298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be88306987c48fa88684d9ac9441959'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:46.171194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.172694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.183740 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 754/1511] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:46.184648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.186007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.197251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9d13a-2bb4-45b9-a1f9-126dd2d26140 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.202398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.203250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.203742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.204030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.205860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.221751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.224293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.236006 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 755/1512] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:46.237086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.238772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.252930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f451-c5c8-4301-b30a-cdfd83c42c27 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.258858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be88306987c48fa88684d9ac9441959'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:46.259174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.259849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.260151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.260415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.261071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.262738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.274709 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 756/1513] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:46.287616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be88306987c48fa88684d9ac9441959'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:46.287937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.287937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.288121 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 758/1514] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:45 2022] DELETE /identity/v3/users/9be88306987c48fa88684d9ac9441959 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:46.288121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.304833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32499-5a62-44da-9932-4bffc6dff1bb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.359903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8f49-3a54-4281-b342-180c05f38ab0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.365512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.366340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.366623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.366910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] PATH_INFO: `/v3/projects/67cdd608eb694a82b9180ef05795f672/users/21aaa4a59baf43898e8e7220da337a42/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.369509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=67cdd608eb694a82b9180ef05795f672, user_id=21aaa4a59baf43898e8e7220da337a42, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.372320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b6d48-3e96-43c0-a850-2c8aa8d51bbd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.376232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.377266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.377591 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.377871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] PATH_INFO: `/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5/users/bd0ade11b6b84c0e9cfdc9eb0025f966/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.386957 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 759/1515] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:46 2022] PUT /identity/v3/projects/67cdd608eb694a82b9180ef05795f672/users/21aaa4a59baf43898e8e7220da337a42/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:46.397330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.397627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.398411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.399252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa81d657a5d4a94b1fcd5a505c2acf5, user_id=bd0ade11b6b84c0e9cfdc9eb0025f966, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.402419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.428483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] MFA Rules not processed for user `21aaa4a59baf43898e8e7220da337a42`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:46.429741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.476268 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 757/1516] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:46 2022] PUT /identity/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5/users/bd0ade11b6b84c0e9cfdc9eb0025f966/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:46.480691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.513973 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 760/1517] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:46.541494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.547539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.565586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b7f0-c369-4ed5-a1a0-c8d3976de210 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.571212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.572104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.576062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.576062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.576062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.576062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.592595 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 758/1518] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:09:46.601077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.625325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a600f-2352-468b-93e4-ac7b4cd882f5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.653505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384984-5344-4a76-a1b8-34d18e015c83 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:46.668175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.668711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.669631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.669742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] PATH_INFO: `/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5/users/bd0ade11b6b84c0e9cfdc9eb0025f966/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.679078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa81d657a5d4a94b1fcd5a505c2acf5, user_id=bd0ade11b6b84c0e9cfdc9eb0025f966, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.682519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb0da-e85f-41f2-b0e3-34688403a24b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.699640 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 759/1519] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:46 2022] PUT /identity/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5/users/bd0ade11b6b84c0e9cfdc9eb0025f966/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:46.710119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.711143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.712280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.728447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:46.757423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] MFA Rules not processed for user `bd0ade11b6b84c0e9cfdc9eb0025f966`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:46.759543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:46.776339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:46.777153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:46.777496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:46.777866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:46.901728 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 760/1520] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:46.908977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:46.910670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:46.928009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:46.934676 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 761/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:46.937381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.037961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4774e9-e82a-420f-aa3f-f135409c4f6e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.041439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.041999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.042593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.042878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.043156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.053711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30c97-fc47-45f1-9882-89cce6e6e5b3 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.057569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.061779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.061985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.061985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.061985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.061985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.076115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.077228 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 761/1522] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:47.103906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.103906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.124479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:47.162206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.163790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.166161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787d304-c08d-4635-8e04-f065a59974cd service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.170767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.171405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.171726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.172003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] PATH_INFO: `/v3/projects/6d67aab66bdc4e4884a738899629b3b6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.173193 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 762/1523] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:47.173819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.175677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.176220 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d67aab66bdc4e4884a738899629b3b6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.183538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.193293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6f67d-5026-4234-9b97-6148f37c3f17 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.195525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.196767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.197051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.197329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] PATH_INFO: `/v3/projects/4478faebdde8453a9ef1ac61018a9fdc` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.198462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] RBAC: Authorizing `identity:delete_project(project_id=4478faebdde8453a9ef1ac61018a9fdc)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.200443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.330957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4478faebdde8453a9ef1ac61018a9fdc', 'request_id': 'req-3e444685-15cc-4695-82bc-411cc24b64da'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.337789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4478faebdde8453a9ef1ac61018a9fdc', 'request_id': 'req-3e444685-15cc-4695-82bc-411cc24b64da'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.338063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d67aab66bdc4e4884a738899629b3b6', 'request_id': 'req-d1f06f99-3c86-4322-9877-dfdc9c639815'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.344031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d67aab66bdc4e4884a738899629b3b6', 'request_id': 'req-d1f06f99-3c86-4322-9877-dfdc9c639815'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.345063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4478faebdde8453a9ef1ac61018a9fdc', 'request_id': 'req-3e444685-15cc-4695-82bc-411cc24b64da'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.452871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d67aab66bdc4e4884a738899629b3b6', 'request_id': 'req-d1f06f99-3c86-4322-9877-dfdc9c639815'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.466846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] The token cache is being invalidate because project 6d67aab66bdc4e4884a738899629b3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:47.467354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.468089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.485874 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 762/1524] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:47 2022] DELETE /identity/v3/projects/6d67aab66bdc4e4884a738899629b3b6 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:47.485874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.485874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.537490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] The token cache is being invalidate because project 4478faebdde8453a9ef1ac61018a9fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:47.537797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.538552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.540202 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 763/1525] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:47 2022] DELETE /identity/v3/projects/4478faebdde8453a9ef1ac61018a9fdc => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:47.541112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.542415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.549798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06f99-3c86-4322-9877-dfdc9c639815 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.554483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.555699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.555977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.556247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] PATH_INFO: `/v3/projects/d1c9f837797e4695979e53b7e70336d2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.557377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c9f837797e4695979e53b7e70336d2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.560080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.606994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e444685-15cc-4695-82bc-411cc24b64da admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.656256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.656823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.657161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.657369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.659292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.720627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.721413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c9f837797e4695979e53b7e70336d2', 'request_id': 'req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.722099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.729041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c9f837797e4695979e53b7e70336d2', 'request_id': 'req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.732979 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 764/1526] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:47.734922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.735073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.749433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc7182-fdf9-4f6d-b57e-8bea9785baad service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.806083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c9f837797e4695979e53b7e70336d2', 'request_id': 'req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.817294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.817959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.818245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.818522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] PATH_INFO: `/v3/users/bb63722318b14ef08c78595a2dff515c` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.819775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb63722318b14ef08c78595a2dff515c)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.821567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] The token cache is being invalidate because project d1c9f837797e4695979e53b7e70336d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:47.821802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:47.822265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.823134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:47.824908 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 763/1527] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:47 2022] DELETE /identity/v3/projects/d1c9f837797e4695979e53b7e70336d2 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:47.825911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:47.827455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:47.844984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b771cc2-3e1e-4d1d-9fdd-8b21b5ee52b8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:47.852954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:47.853607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:47.853855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:47.854122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] PATH_INFO: `/v3/users/eca2326a391e458283b88ee9ec4cf374` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:47.855337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca2326a391e458283b88ee9ec4cf374)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:47.857430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.072871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb63722318b14ef08c78595a2dff515c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.073562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb63722318b14ef08c78595a2dff515c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.074038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca2326a391e458283b88ee9ec4cf374'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.074927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca2326a391e458283b88ee9ec4cf374'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.083980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca2326a391e458283b88ee9ec4cf374'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.084779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca2326a391e458283b88ee9ec4cf374'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.090456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb63722318b14ef08c78595a2dff515c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.102781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-084d07b5-66c2-45fa-b584-64666a85de55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca2326a391e458283b88ee9ec4cf374'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.178321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb63722318b14ef08c78595a2dff515c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.187829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-620de7ab-bf90-43a6-92d9-a4ff51dbf9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb63722318b14ef08c78595a2dff515c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.195412 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 765/1528] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:47 2022] DELETE /identity/v3/users/bb63722318b14ef08c78595a2dff515c => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:48.196766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.197451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.197451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.220790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:48.222114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:48.270225 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 764/1529] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:47 2022] DELETE /identity/v3/users/eca2326a391e458283b88ee9ec4cf374 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:48.293573 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 766/1530] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:48.301835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796777-a561-4635-9e5f-efd947fab939 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.304722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796777-a561-4635-9e5f-efd947fab939 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.304722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796777-a561-4635-9e5f-efd947fab939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.328242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-73796777-a561-4635-9e5f-efd947fab939 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:48.330140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-73796777-a561-4635-9e5f-efd947fab939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:48.399662 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 765/1531] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:48.402573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.413304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e5d0e-4755-4862-b711-a36e2270ad3f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.418319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.419043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.419564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.419825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.420592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:48.421421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.422845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.436192 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 767/1532] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:09:48.440404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73796777-a561-4635-9e5f-efd947fab939 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.450694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73796777-a561-4635-9e5f-efd947fab939 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.457977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.458695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.458984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.459261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.459973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.462585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.475295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:48.582614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.582792 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 766/1533] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:48 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:48.600769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7991e-6ccf-4c20-931c-2c79b0afafd6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.687371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.688139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.688498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.688848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.689622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.691725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.778429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.782476 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 768/1534] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:48 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:48.782662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:48.783782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.794639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3ee9-2c2e-4105-97af-9ecc3eed9501 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.797107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37820d0-3d9d-4711-83d3-bf38d3569b00 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.800065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.800808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.801200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.801549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] PATH_INFO: `/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.803011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76705e3b237142e18ba5bdb2cbb10f6b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.804010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.804818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.805696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.806181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.806439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.807080 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.809495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.824950 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 769/1535] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:09:48.826144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:48.828162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:48.849669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeccfc0-4f06-4a39-9562-d6f155a93fd9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:48.853310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:48.853955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:48.854264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:48.854546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] PATH_INFO: `/v3/projects/00de37c042ab410880d497bc938bc040` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:48.855724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00de37c042ab410880d497bc938bc040)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:48.857741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:48.910942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:48.932714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76705e3b237142e18ba5bdb2cbb10f6b', 'request_id': 'req-85b9b784-7cef-44eb-8d83-a30694e5e326'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.939597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76705e3b237142e18ba5bdb2cbb10f6b', 'request_id': 'req-85b9b784-7cef-44eb-8d83-a30694e5e326'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.977891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00de37c042ab410880d497bc938bc040', 'request_id': 'req-97d122de-ff2a-4d06-a6a5-deab51bb8378'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.983881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00de37c042ab410880d497bc938bc040', 'request_id': 'req-97d122de-ff2a-4d06-a6a5-deab51bb8378'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:48.994582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00de37c042ab410880d497bc938bc040', 'request_id': 'req-97d122de-ff2a-4d06-a6a5-deab51bb8378'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.029431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76705e3b237142e18ba5bdb2cbb10f6b', 'request_id': 'req-85b9b784-7cef-44eb-8d83-a30694e5e326'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.043714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] The token cache is being invalidate because project 76705e3b237142e18ba5bdb2cbb10f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:49.044090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.044894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.046852 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 767/1536] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:48 2022] DELETE /identity/v3/projects/76705e3b237142e18ba5bdb2cbb10f6b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:49.047483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.048909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.090231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] The token cache is being invalidate because project 00de37c042ab410880d497bc938bc040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:49.090785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.091836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-97d122de-ff2a-4d06-a6a5-deab51bb8378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:49.093615 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 770/1537] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:48 2022] DELETE /identity/v3/projects/00de37c042ab410880d497bc938bc040 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:49.095035 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8876f3d-2312-4851-956f-abed514bfe48 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.095354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8876f3d-2312-4851-956f-abed514bfe48 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.095685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8876f3d-2312-4851-956f-abed514bfe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.119386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d8876f3d-2312-4851-956f-abed514bfe48 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:49.120678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d8876f3d-2312-4851-956f-abed514bfe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.126807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b784-7cef-44eb-8d83-a30694e5e326 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.189451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.190076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.190377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.190652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.192969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.196925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd3a48-8e67-42ee-b336-920177d2dbe7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.199570 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 771/1538] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:49.201714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30437-65aa-4788-b935-233ee848571a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.202065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30437-65aa-4788-b935-233ee848571a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.202403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30437-65aa-4788-b935-233ee848571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.214485 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 768/1539] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:49.219034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.219340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.219644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.245618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:49.245618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.253637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-56a30437-65aa-4788-b935-233ee848571a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:49.254861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-56a30437-65aa-4788-b935-233ee848571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.263911 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 769/1540] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:49.264354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.266704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.280710 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 772/1541] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:49.281598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a30437-65aa-4788-b935-233ee848571a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.293955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088c37c-6172-40cd-91b4-cbe21ee7350c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.298435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a30437-65aa-4788-b935-233ee848571a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.304544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.308776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.308776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.308776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] PATH_INFO: `/v3/projects/bba669eb2db44f6eb7c3f2fdb419c5c3/users/7917e9f7c9b544b8bf17c66da1f949a8/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.310363 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.311010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.311294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.311570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.312259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:49.312565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.314213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddcb2b-4cfd-47be-8430-76845a3d53c9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.337466 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 773/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:49.338769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.339061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.339344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.342078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba669eb2db44f6eb7c3f2fdb419c5c3, user_id=7917e9f7c9b544b8bf17c66da1f949a8, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.344926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.374151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:49.375750 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.386208 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 774/1543] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:49.387545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.401781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a095d-76b4-4eb3-bfbe-f340d6c0b1c8 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.406822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.407455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.408306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.408589 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.409246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.413011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3de471-2558-4604-a637-c6bfc95e9026 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.453402 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 770/1544] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:49 2022] PUT /identity/v3/projects/bba669eb2db44f6eb7c3f2fdb419c5c3/users/7917e9f7c9b544b8bf17c66da1f949a8/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:49.454979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.510679 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 775/1545] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:49.513157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.513674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.514179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.543303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c88b8a-4c89-45e4-ba7a-47020dde081e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.558633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] MFA Rules not processed for user `7917e9f7c9b544b8bf17c66da1f949a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:49.559901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.646029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.646956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.647149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.660232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.660232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:49.660232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.660232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.661376 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 776/1546] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:49.661376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.683790 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 771/1547] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:49.683973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.690028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bea52-d778-4bbb-b9e5-ce0b86f1778d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.690762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:49.696881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.705421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.705421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.705421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.705421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.705421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.807003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2aa0-c122-45bc-84c7-cd909aef4e9d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.812639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.813909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.814438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.814717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.823686 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 777/1548] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:49.826789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.827465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.828425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.841692 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 772/1549] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:49.843368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.857340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a02f9-5172-4777-a7ea-3ef79c11435e service glance] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.861229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215cc-3283-4112-80b2-b16c27a48e0e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.866644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.867514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.867976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.868229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.868924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.870687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.881177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.881818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.882110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.882396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.883606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.886534 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 773/1550] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:49.890250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.890454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.906325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae40d7-c5dd-43c4-a4a6-8e8220f9483f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.918535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.919162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.919444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.919755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.920410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:49.926908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:49.952218 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 774/1551] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:49.953629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:49.974267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c081c3-6d6e-4567-96ee-ff315b38d1e0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:49.982600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:49.983438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:49.983550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:49.983848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] PATH_INFO: `/v3/projects/1fa989b4a6dc4e59a700894346b91ec4/users/93f98441d1ff46ed8e16f6f7659f248c/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:49.998245 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 778/1552] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:49 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:50.002743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.025543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa989b4a6dc4e59a700894346b91ec4, user_id=93f98441d1ff46ed8e16f6f7659f248c, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.028996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b7679-9f9c-4791-a434-d6df2a047c49 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.114180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25cece-3ffd-4e1b-af39-868aba1f0886 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.119398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.120105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.120430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.120740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.121388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.128259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.159680 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 775/1553] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:49 2022] PUT /identity/v3/projects/1fa989b4a6dc4e59a700894346b91ec4/users/93f98441d1ff46ed8e16f6f7659f248c/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:50.169257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350b33-5f48-467f-9932-7ee06681016c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.169628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350b33-5f48-467f-9932-7ee06681016c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.169913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350b33-5f48-467f-9932-7ee06681016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.175672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:50.215338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-68350b33-5f48-467f-9932-7ee06681016c None None] MFA Rules not processed for user `93f98441d1ff46ed8e16f6f7659f248c`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:50.219867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-68350b33-5f48-467f-9932-7ee06681016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:50.246047 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 779/1554] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:50 2022] POST /identity/v3/users => generated 294 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:50.246961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.260812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc56f0-124c-41e1-9b10-549f23cc9358 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.434627 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 776/1555] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:50.436159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68350b33-5f48-467f-9932-7ee06681016c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.439872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-68350b33-5f48-467f-9932-7ee06681016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:50.512409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.512933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.513217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.513492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.514144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.515795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.529541 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 780/1556] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:50.529541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.535042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68350b33-5f48-467f-9932-7ee06681016c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.536660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978804-b3ea-471b-b24f-a3eac5386ef0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.539794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.540448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.540737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.541017 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] PATH_INFO: `/v3/users/4ebc13e4a9b4411a9b962587643abaf3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.541262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.541857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.542161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.542284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebc13e4a9b4411a9b962587643abaf3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.542455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.543059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.544134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.544976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.865239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:50.872573 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 781/1557] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:50 2022] POST /identity/v3/projects => generated 387 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:50.876720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.938441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a2e4-42f7-4b22-af00-3efabe58e8db admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.943470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.944118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.944406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.944680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.945329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.946967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:50.957425 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 782/1558] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:50.958234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:50.968164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c469773-0cdd-4741-82e9-1cf8782f7387 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:50.972741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:50.973330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:50.973613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:50.973888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:50.974512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:50.976515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.027403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebc13e4a9b4411a9b962587643abaf3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:51.028356 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebc13e4a9b4411a9b962587643abaf3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:51.035883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebc13e4a9b4411a9b962587643abaf3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:51.036891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebc13e4a9b4411a9b962587643abaf3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:51.050126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebc13e4a9b4411a9b962587643abaf3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:51.096975 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 783/1559] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:50 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:51.097798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.099482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.112456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c40bf-5e18-4ea0-97e5-cfa166d06420 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.117340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.118009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.118291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.118733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] PATH_INFO: `/v3/projects/aa4bdec6d03b414882da5ca42b0a2dea/users/47b44e4d5a5344e2938d7551626ab287/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.126890 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 777/1560] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:50 2022] DELETE /identity/v3/users/4ebc13e4a9b4411a9b962587643abaf3 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:51.128840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.135222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.149911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4bdec6d03b414882da5ca42b0a2dea, user_id=47b44e4d5a5344e2938d7551626ab287, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.153073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.205057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce40d71-94f2-4ed8-b095-24197ba724e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.210175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.210820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.211715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae46c9-f9f3-4a84-90f8-8c70549e8265 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 778/1561] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.226045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.227361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.245694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:51.246910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.257116 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 779/1562] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:51.257116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.271651 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 784/1563] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:51 2022] PUT /identity/v3/projects/aa4bdec6d03b414882da5ca42b0a2dea/users/47b44e4d5a5344e2938d7551626ab287/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:51.271974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944a4d-12e3-4847-a0ef-93bc039745ec None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.273069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.281722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.346465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:51.407468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.411903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.417966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.417966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] PATH_INFO: `/v3/projects/a41b8fb511104bfa890e8915aed2dfa0/users/1988188d9739454bbb8c08312a2a6df9/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.442589 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a631d-fa44-460b-bc21-4b87c972ba9f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.446984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41b8fb511104bfa890e8915aed2dfa0, user_id=1988188d9739454bbb8c08312a2a6df9, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.454574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e1f4b6-b1b5-498b-9470-4aed0b1eeb66 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.464816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.466102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.466857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.467627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.519259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.527465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.556286 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 780/1564] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:51 2022] PUT /identity/v3/projects/a41b8fb511104bfa890e8915aed2dfa0/users/1988188d9739454bbb8c08312a2a6df9/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:51.562119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.566524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.566524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.585969 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 785/1565] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:51.594848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.625003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] MFA Rules not processed for user `47b44e4d5a5344e2938d7551626ab287`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:51.628914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947b03-af47-41b5-8011-8ac2a77544df service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.629078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.747740 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 781/1566] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:51.750726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.757025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:51.763915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.783763 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 786/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:51.785441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:51.796031 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea719819-2d2f-4ca6-b9d2-37504eeeaaaf admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.805460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.806759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.806943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.807152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] PATH_INFO: `/v3/projects/a41b8fb511104bfa890e8915aed2dfa0/users/1988188d9739454bbb8c08312a2a6df9/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.809295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41b8fb511104bfa890e8915aed2dfa0, user_id=1988188d9739454bbb8c08312a2a6df9, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.813632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7b67e-62af-4cda-8b38-92f90481fafb admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:51.830208 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 787/1568] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:51 2022] PUT /identity/v3/projects/a41b8fb511104bfa890e8915aed2dfa0/users/1988188d9739454bbb8c08312a2a6df9/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:51.843292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.843707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.843989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.862604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19b221-eceb-4742-8766-0eca93f224aa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:51.875303 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] MFA Rules not processed for user `1988188d9739454bbb8c08312a2a6df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:51.876567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:51.955207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:51.956167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:51.956521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:51.957478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:51.958337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:51.962992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.038384 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 788/1569] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:52.057774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.059566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.064842 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 782/1570] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:51 2022] POST /identity/v3/projects => generated 387 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:52.069352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.174521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb3940-4aa9-4e94-a095-5f8fd7450dc1 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.190431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6854a5-2531-4596-b265-58a7e5f52976 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.196290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.197107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.197747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.198216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.199146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.201603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973e4de-398b-41f3-a97f-f3afa19212ba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.285895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.286506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.286789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.287061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.305890 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 783/1571] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:52 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:52.308366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36871-2877-43d9-9e1c-5e2925b50e50 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.308366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36871-2877-43d9-9e1c-5e2925b50e50 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.308366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36871-2877-43d9-9e1c-5e2925b50e50 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.310841 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 784/1572] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:09:52.312072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36871-2877-43d9-9e1c-5e2925b50e50 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.328054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36871-2877-43d9-9e1c-5e2925b50e50 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.341503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.342331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.342613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.343310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.344057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.345923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dee3a5-f78f-4ca1-9961-a05c86f3312e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.360378 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 785/1573] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:52.361952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.362273 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.362641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.397700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:52.399218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.428735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.433213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.481612 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 789/1574] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:52.484320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.511090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6dd57-a799-48b7-ab4f-54dc663864ea service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.517008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.517967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.518144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.518418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.519148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.520848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.530176 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 786/1575] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:09:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:52.530176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.530176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.532917 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 790/1576] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:52.540775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.553126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a6b90-a8dc-4f49-8f6c-70e3dc714b1b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.555741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87cc0c-7424-4a78-8e88-91969a81b844 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.561582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.568436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.568436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.568682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.568790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] PATH_INFO: `/v3/projects/193926ed37184666859980c480f36157` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.568870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.568870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.568870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] PATH_INFO: `/v3/projects/1e38e7d52fbe45f4a21bfc65fabd07bb/users/5aeebbd08b62488ab0aa5334408942f6/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.569173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=193926ed37184666859980c480f36157)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.569173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.607891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e38e7d52fbe45f4a21bfc65fabd07bb, user_id=5aeebbd08b62488ab0aa5334408942f6, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:52.625998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:52.739488 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 791/1577] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:52 2022] PUT /identity/v3/projects/1e38e7d52fbe45f4a21bfc65fabd07bb/users/5aeebbd08b62488ab0aa5334408942f6/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:52.740834 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:52.742634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.748052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193926ed37184666859980c480f36157', 'request_id': 'req-2a047c92-5045-47c2-8f20-19cfd271bc3e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:52.765398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193926ed37184666859980c480f36157', 'request_id': 'req-2a047c92-5045-47c2-8f20-19cfd271bc3e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:52.838907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193926ed37184666859980c480f36157', 'request_id': 'req-2a047c92-5045-47c2-8f20-19cfd271bc3e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:52.844652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8533-929c-4ff3-92da-da6265062768 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:52.854147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] The token cache is being invalidate because project 193926ed37184666859980c480f36157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:52.854654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:52.856526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2a047c92-5045-47c2-8f20-19cfd271bc3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:52.860913 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 787/1578] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:52 2022] DELETE /identity/v3/projects/193926ed37184666859980c480f36157 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:52.863684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.869194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.869773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.911619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] MFA Rules not processed for user `5aeebbd08b62488ab0aa5334408942f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:52.913189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.965308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:52.966081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:52.966502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:52.967175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:52.969809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:52.994018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:53.006415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.009383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.023403 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 792/1579] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:53.024685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.026247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:53.037028 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 788/1580] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:53.044991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.046698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:53.175405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1074a-e810-4b0c-a4f2-d0e4352b6c4d service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:53.176044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a7ff4-3d12-4503-9122-871369b4530f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:53.182671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:53.183321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:53.183776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:53.183971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:53.184689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.187534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:53.193069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:53.193069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:53.193069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] PATH_INFO: `/v3/users/21aaa4a59baf43898e8e7220da337a42` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:53.193376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.193471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] RBAC: Authorizing `identity:delete_user(user_id=21aaa4a59baf43898e8e7220da337a42)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.210714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.291632 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 789/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:53 2022] POST /identity/v3/projects => generated 385 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:53.292999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.412678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffd64-9120-4bc2-8fdb-0cec5b7b6bb7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:53.430486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:53.431223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:53.432620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:53.433050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:53.433986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.436958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.564360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21aaa4a59baf43898e8e7220da337a42'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:53.565006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21aaa4a59baf43898e8e7220da337a42'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:53.589541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21aaa4a59baf43898e8e7220da337a42'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:53.603958 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 790/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:53 2022] POST /identity/v3/users => generated 299 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:53.605450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.607469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:53.621162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68397-7ac4-43ef-8b44-6c951871ba82 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:53.645981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:53.648449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:53.648449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:53.648449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:53.648898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.650736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.663325 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 791/1583] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:53.670757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.684965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef6a2-e9e2-4a3a-84ea-8a168907cbb5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:53.690723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:53.691399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:53.691748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:53.692020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] PATH_INFO: `/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf/users/4ebcbcaaef75444094b1753657ce7e0a/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:53.709660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21aaa4a59baf43898e8e7220da337a42'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:53.713670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3004db3ba714f7f8bac1a8bfc511daf, user_id=4ebcbcaaef75444094b1753657ce7e0a, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:53.719725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21aaa4a59baf43898e8e7220da337a42'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:53.724109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:53.735651 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 793/1584] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:53 2022] DELETE /identity/v3/users/21aaa4a59baf43898e8e7220da337a42 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:53.834692 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 792/1585] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:53 2022] PUT /identity/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf/users/4ebcbcaaef75444094b1753657ce7e0a/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:53.838392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:53.935231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f73e82-42a8-40d4-9b9e-49e2dcdb639f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:54.011946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:54.013871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.014666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.014954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.016137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:54.018350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:54.018508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72b417-dae6-4cc1-b212-915971d95748 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:54.036443 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 794/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:54.037785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:54.061901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee559-06ee-4bd2-8584-a80af64f949d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:54.078252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:54.081059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.081059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.082888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] PATH_INFO: `/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf/users/4ebcbcaaef75444094b1753657ce7e0a/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.087900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3004db3ba714f7f8bac1a8bfc511daf, user_id=4ebcbcaaef75444094b1753657ce7e0a, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:54.091878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:54.122576 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 793/1587] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:54 2022] PUT /identity/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf/users/4ebcbcaaef75444094b1753657ce7e0a/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:54.134579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.134913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.135250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.145037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:54.186886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] MFA Rules not processed for user `4ebcbcaaef75444094b1753657ce7e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:54.188172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.290889 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 795/1588] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:54.301239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:54.303411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.304257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:54.307666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.382359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b5235-3893-4c9e-84ac-e6b05551fc2f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:54.387418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:54.393533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.393533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.393533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.398976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:54.400660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:54.411464 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 796/1589] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:54.412364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:54.413669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.428686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d322-aa6d-48c3-832d-2170ea29291d service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:54.432662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753341c-74fa-4302-976d-e6f812b0abaf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:54.434940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:54.435474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.435792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.436071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] PATH_INFO: `/v3/projects/67cdd608eb694a82b9180ef05795f672` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.437223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67cdd608eb694a82b9180ef05795f672)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:54.438312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:54.439111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.439332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:54.439788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.440053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] PATH_INFO: `/v3/users/93f98441d1ff46ed8e16f6f7659f248c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.441546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f98441d1ff46ed8e16f6f7659f248c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:54.443625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:54.708887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67cdd608eb694a82b9180ef05795f672', 'request_id': 'req-7ded5670-b7b9-4f0a-a231-96be4facba71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.716598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67cdd608eb694a82b9180ef05795f672', 'request_id': 'req-7ded5670-b7b9-4f0a-a231-96be4facba71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.721442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f98441d1ff46ed8e16f6f7659f248c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.722298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f98441d1ff46ed8e16f6f7659f248c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.728234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67cdd608eb694a82b9180ef05795f672', 'request_id': 'req-7ded5670-b7b9-4f0a-a231-96be4facba71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.731326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f98441d1ff46ed8e16f6f7659f248c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.732291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f98441d1ff46ed8e16f6f7659f248c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.744923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-259bd709-9912-4bf1-bcaa-32956df930ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f98441d1ff46ed8e16f6f7659f248c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.825989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] The token cache is being invalidate because project 67cdd608eb694a82b9180ef05795f672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:54.826425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.827492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7ded5670-b7b9-4f0a-a231-96be4facba71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:54.829673 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 797/1590] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:54 2022] DELETE /identity/v3/projects/67cdd608eb694a82b9180ef05795f672 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:54.835572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207e369-a008-4411-9ed0-a804f186872a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.835887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207e369-a008-4411-9ed0-a804f186872a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.836709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207e369-a008-4411-9ed0-a804f186872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.862631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8207e369-a008-4411-9ed0-a804f186872a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:54.864389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8207e369-a008-4411-9ed0-a804f186872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.882559 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 794/1591] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:54 2022] DELETE /identity/v3/users/93f98441d1ff46ed8e16f6f7659f248c => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:54.888352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.889885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.889885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:54.918450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:54.919872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:54.971882 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 798/1592] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:54.982046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73903-b245-4905-b4b1-09b383671092 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:54.985690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73903-b245-4905-b4b1-09b383671092 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:54.985690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73903-b245-4905-b4b1-09b383671092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.005819 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 795/1593] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:55.010492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2da73903-b245-4905-b4b1-09b383671092 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:55.011882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2da73903-b245-4905-b4b1-09b383671092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:55.016875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.020662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:55.022735 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 799/1594] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:55.026976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73903-b245-4905-b4b1-09b383671092 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.038819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73903-b245-4905-b4b1-09b383671092 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.048428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.049326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.049757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.050212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.051028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:55.051484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.053277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.070074 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 800/1595] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:55.075279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.094869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f422f86-a3f6-472c-93f8-8fbb67151859 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.096877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.097575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.097865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.098214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.098856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.100839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.111979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f89ade-f239-45cf-bb19-2beedb102f3d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.117247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.118992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.119936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.120375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.132003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.133393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.150924 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 796/1596] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:09:55.201598 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 801/1597] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:55 2022] POST /identity/v3/projects => generated 359 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:55.205891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.216556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cd9b0-66d4-406a-abc3-6055c6009e7f service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.291943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.292579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.292867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.293145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.293804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.295778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.452560 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 797/1598] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:55 2022] POST /identity/v3/users => generated 280 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:55.456919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.475723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09042b-8d18-4e80-bc0f-be9a488e7670 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.481105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.484533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.484533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.484533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.484533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.485121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.501462 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 802/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:55.505797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.526557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cb2-ea51-4074-b46c-01ed769b81d7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.536675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.538520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.540983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.542652 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.543056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.544986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.575945 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 798/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:55.581030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.629359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5318b-6d10-49e0-962e-733803cb3b7a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.646898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.647897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.648920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.651360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] PATH_INFO: `/v3/projects/19eda877b0384a4380a34117a2e2a75e/users/afd84034650b43dc928c30197ceebec1/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.695983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19eda877b0384a4380a34117a2e2a75e, user_id=afd84034650b43dc928c30197ceebec1, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.698852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe20763-b850-4c6f-b8a4-6b823fd6f886 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.755082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:55.757017 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:55.784322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5074b-adb7-48e4-806d-05eec9236020 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:55.800963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:55.801711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.804793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.804793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] PATH_INFO: `/v3/projects/1fa989b4a6dc4e59a700894346b91ec4` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.804793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa989b4a6dc4e59a700894346b91ec4)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:55.805061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 803/1601] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:55 2022] PUT /identity/v3/projects/19eda877b0384a4380a34117a2e2a75e/users/afd84034650b43dc928c30197ceebec1/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:55.805447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:55.815746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:55.816175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:55.816550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:55.886128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] MFA Rules not processed for user `afd84034650b43dc928c30197ceebec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:55.886128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:55.960425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa989b4a6dc4e59a700894346b91ec4', 'request_id': 'req-6da18c91-21d1-4857-b555-cde42d7c58a0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:55.971010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa989b4a6dc4e59a700894346b91ec4', 'request_id': 'req-6da18c91-21d1-4857-b555-cde42d7c58a0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.030269 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 804/1602] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:56.044030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:56.046403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.108889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa989b4a6dc4e59a700894346b91ec4', 'request_id': 'req-6da18c91-21d1-4857-b555-cde42d7c58a0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.124244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] The token cache is being invalidate because project 1fa989b4a6dc4e59a700894346b91ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:56.124802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.125536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.127739 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 799/1603] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:55 2022] DELETE /identity/v3/projects/1fa989b4a6dc4e59a700894346b91ec4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:56.131748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:56.133973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.149496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:56.153189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c807c98-deda-43e2-a5f1-4050983399f5 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:56.228949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18c91-21d1-4857-b555-cde42d7c58a0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:56.234466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:56.235038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:56.235327 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:56.235682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] PATH_INFO: `/v3/users/1988188d9739454bbb8c08312a2a6df9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:56.236831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1988188d9739454bbb8c08312a2a6df9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:56.238984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:56.246335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:56.248090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:56.248567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:56.249025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:56.252591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.355660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:56.357128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:56.366809 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 805/1604] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:56.368226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:56.369541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.380362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca8a33-2256-45ec-b566-d26efb8a99ed service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:56.384822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:56.385433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:56.385715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:56.385988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:56.387792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.408824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:56.474864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1988188d9739454bbb8c08312a2a6df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.475612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1988188d9739454bbb8c08312a2a6df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.483388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:56.484901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:56.487458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1988188d9739454bbb8c08312a2a6df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.488432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1988188d9739454bbb8c08312a2a6df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.496914 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 806/1605] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:56.503772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de9606f0-a97f-4335-9ca0-32be5178f650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1988188d9739454bbb8c08312a2a6df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.514321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:56.515839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:56.530965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85edb64-693a-416e-913e-520b5d3d8a18 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:56.542316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:56.542316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:56.542316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:56.542316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] PATH_INFO: `/v3/users/afd84034650b43dc928c30197ceebec1` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:56.542665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd84034650b43dc928c30197ceebec1)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:56.542665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:56.635471 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 800/1606] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:56 2022] DELETE /identity/v3/users/1988188d9739454bbb8c08312a2a6df9 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:56.766791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd84034650b43dc928c30197ceebec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.767719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd84034650b43dc928c30197ceebec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.781690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd84034650b43dc928c30197ceebec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.863453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd84034650b43dc928c30197ceebec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.876057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd84034650b43dc928c30197ceebec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:56.903529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 807/1607] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:56 2022] DELETE /identity/v3/users/afd84034650b43dc928c30197ceebec1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:09:56.903781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:56.903781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:56.903781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:56.925420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:56.930479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.005043 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 801/1608] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:57.016790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.018411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.132200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb916e5-1ed8-493a-a9e3-99569ccd1887 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:57.155765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:57.156961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.157319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.157704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.171375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:57.179578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51826a-0771-45c4-ace7-9e0cc07b7d41 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:57.214044 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 808/1609] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:57.252844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.256428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.283479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae789e-4dc7-44fd-97d9-068504f4907b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:57.298013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:57.308872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.308872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.308872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] PATH_INFO: `/v3/projects/a41b8fb511104bfa890e8915aed2dfa0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.308872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41b8fb511104bfa890e8915aed2dfa0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:57.308872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:57.472691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41b8fb511104bfa890e8915aed2dfa0', 'request_id': 'req-4ff18336-43d6-44fa-a4ee-4c045a383e80'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.480840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41b8fb511104bfa890e8915aed2dfa0', 'request_id': 'req-4ff18336-43d6-44fa-a4ee-4c045a383e80'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.566659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41b8fb511104bfa890e8915aed2dfa0', 'request_id': 'req-4ff18336-43d6-44fa-a4ee-4c045a383e80'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.582130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] The token cache is being invalidate because project a41b8fb511104bfa890e8915aed2dfa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:57.582402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.583099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4ff18336-43d6-44fa-a4ee-4c045a383e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.584657 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 802/1610] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:57 2022] DELETE /identity/v3/projects/a41b8fb511104bfa890e8915aed2dfa0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:57.590307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.590919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.590919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.613531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:57.615133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.691379 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 809/1611] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:57.699803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.700089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.701234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.728302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:57.729698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.753114 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 803/1612] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:57.757404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.768290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6167b-e697-4e31-8072-bb77f06bf92b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:57.772993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:57.773711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.773908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.774206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.774885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:57.775186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:57.776824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:57.789457 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 810/1613] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:09:57.794031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.803746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606dc3d-4ebf-4957-b5ab-3f92e9ecef15 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:57.809885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:57.810551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.810834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.811110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.811783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:57.813858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:57.825781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.827861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:57.842576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce34709-3b60-443a-99dc-13486864a174 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:57.847633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:57.848362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:57.848942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:57.849224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] PATH_INFO: `/v3/projects/19eda877b0384a4380a34117a2e2a75e` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:57.850433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19eda877b0384a4380a34117a2e2a75e)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:57.853503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:57.893671 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 804/1614] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:57 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:57.897519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:57.949585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19eda877b0384a4380a34117a2e2a75e', 'request_id': 'req-219a19c5-89ac-43b0-be3b-aad5d059538b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.958294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19eda877b0384a4380a34117a2e2a75e', 'request_id': 'req-219a19c5-89ac-43b0-be3b-aad5d059538b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.968513 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19eda877b0384a4380a34117a2e2a75e', 'request_id': 'req-219a19c5-89ac-43b0-be3b-aad5d059538b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:57.987692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ab00-24b3-4a45-8b82-6a9e42871fbd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.082870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] The token cache is being invalidate because project 19eda877b0384a4380a34117a2e2a75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:09:58.083402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:58.084585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-219a19c5-89ac-43b0-be3b-aad5d059538b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:58.087090 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 811/1615] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:57 2022] DELETE /identity/v3/projects/19eda877b0384a4380a34117a2e2a75e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:58.093425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd5c8-a927-4487-8bd4-f8e647302cb7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.093814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd5c8-a927-4487-8bd4-f8e647302cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.094318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd5c8-a927-4487-8bd4-f8e647302cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.105520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.106269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.106518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.106753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.107422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.109642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.125449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e74bd5c8-a927-4487-8bd4-f8e647302cb7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:58.126673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e74bd5c8-a927-4487-8bd4-f8e647302cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:58.136396 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 812/1616] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:58.144238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.144521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.145533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.167911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:58.169178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:58.178187 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 813/1617] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:58.183145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.193521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88909913-e04a-4a30-b6cc-12f61a22a4aa None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.198707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.201176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.201492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.201830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.202585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:09:58.202897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.205075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.219196 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 805/1618] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:57 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:58.220793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.222986 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 814/1619] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:09:58.224054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:58.225141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:58.228734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.240912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e6429-61c5-42be-8f99-a031883b7b4d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.241771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a128-fb1f-4157-97c9-e63cd21377f6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.246582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.247226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.247510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.247819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.248070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.248917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.249023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.249023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.249936 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.250227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.251102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.252245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.265381 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 806/1620] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:58.272256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.285421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2128-d769-4ef7-acae-10fe638ab45c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.291522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.297644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.297644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.297644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.297644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.297644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.320526 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 807/1621] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:58.321041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:58.321728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.345026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d863-732d-47b7-a976-bf9916852e0b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.360792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.361581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.361953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.362441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] PATH_INFO: `/v3/projects/5714394086f24b9fa6a155436ece02a6/users/0a1decbdedc141789937fef1b70495ff/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.385696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5714394086f24b9fa6a155436ece02a6, user_id=0a1decbdedc141789937fef1b70495ff, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.387128 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 815/1622] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:58 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:58.391391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.392622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b829b0-450b-4fff-9e33-afa8a6cd5d0e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.480122 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 808/1623] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:58 2022] PUT /identity/v3/projects/5714394086f24b9fa6a155436ece02a6/users/0a1decbdedc141789937fef1b70495ff/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:58.489322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.497655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.497655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.501381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4dad1-fad5-46fe-8fb0-61f989c1ec03 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.546634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] MFA Rules not processed for user `0a1decbdedc141789937fef1b70495ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:58.548890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:58.592708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.599771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.599771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.599771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.599771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.600788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.643250 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 809/1624] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:58.652883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.654382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:58.741289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287ad49-a13e-4b5f-82ea-7006307a7e39 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.745624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.746414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.746824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.747173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.757635 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 816/1625] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:58 2022] POST /identity/v3/users => generated 293 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:58.761824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.764928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.766748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.780211 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 810/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:58.782957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6081b8d-a570-4e21-bf28-7437425c58d3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.800095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.800307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.800534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.800766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.801423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.803030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.811033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.816912 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 817/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:09:58.821353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.840558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbc59f-97a6-4439-b4fe-6c97528768c8 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.842693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed524415-a871-40d8-b813-f8e59e834095 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.848662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.852031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.852653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.852831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] PATH_INFO: `/v3/users/0a1decbdedc141789937fef1b70495ff` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.854374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1decbdedc141789937fef1b70495ff)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.856225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.856918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.857245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.857991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.858428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.860008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.861712 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:58.874892 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 818/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:09:58.883987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:58.896979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e4507-00fb-4f1e-8a05-85497177eb13 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:58.904122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:58.904957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:58.905322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:58.905614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] PATH_INFO: `/v3/projects/3fd7c367b3b84e8583949ed69bbaadee/users/0e881f186b354b2585afdcc355088348/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:58.924357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd7c367b3b84e8583949ed69bbaadee, user_id=0e881f186b354b2585afdcc355088348, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:58.927471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4df425-11e9-465a-8943-b69f8a137ed2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.024430 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 819/1629] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:58 2022] PUT /identity/v3/projects/3fd7c367b3b84e8583949ed69bbaadee/users/0e881f186b354b2585afdcc355088348/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:59.036147 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.036430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.036704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.064520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] MFA Rules not processed for user `0e881f186b354b2585afdcc355088348`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:59.065766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.091374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1decbdedc141789937fef1b70495ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:59.092392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1decbdedc141789937fef1b70495ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:59.100046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1decbdedc141789937fef1b70495ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:59.100713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1decbdedc141789937fef1b70495ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:59.116166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f70eb8f0-7bc7-4d04-b7c2-6e98f279257a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1decbdedc141789937fef1b70495ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:09:59.156715 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 820/1630] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:09:59.161565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.163739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.212390 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 811/1631] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:09:58 2022] DELETE /identity/v3/users/0a1decbdedc141789937fef1b70495ff => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:59.217616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.218638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.218882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.241825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:59.244578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.250514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b6b47-db39-44df-ae47-d630a1d62fa9 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.355079 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 812/1632] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:09:59.356480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.358739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.368395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.369366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.369678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.370117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.370931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.373315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.454095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817d7d-8caa-4fe2-8123-654c3e811b04 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.457979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.458663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.458952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.459231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.469522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.470957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.481291 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 813/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:59.499835 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 821/1634] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:09:59 2022] POST /identity/v3/projects => generated 385 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:09:59.503812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.514291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a5558-e38c-4f59-ac1d-0cb9caf9b1a0 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.579654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.580414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.580698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.580975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.581672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.583657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.690120 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 814/1635] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:09:59 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:09:59.691417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.702322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc9c5-e20b-486f-b72e-4a0afd433f2b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.707270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.707924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.708212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.708524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.709222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.710827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.735681 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 822/1636] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:59.739681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.751076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114879-7c82-4c42-946c-0b3b2439580e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.756163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.756802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.757088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.757367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.758069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.759876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bdd93-ba12-4fb7-9d27-4049a99e37a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.771256 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 815/1637] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:09:59.776516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.787780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ea63-f18b-40e2-a3dd-f6895dd175ad admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:59.792972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:59.793648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.793924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.794195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] PATH_INFO: `/v3/projects/cb5afdba437943dcaaf0018f0960f43f/users/0f126159c3e84635989970a49a4cf81a/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.810799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5afdba437943dcaaf0018f0960f43f, user_id=0f126159c3e84635989970a49a4cf81a, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:59.813552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:59.879250 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 823/1638] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:09:59 2022] PUT /identity/v3/projects/cb5afdba437943dcaaf0018f0960f43f/users/0f126159c3e84635989970a49a4cf81a/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:09:59.890393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:59.891188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:59.891654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:59.924744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:59.927096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.944252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] MFA Rules not processed for user `0f126159c3e84635989970a49a4cf81a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:09:59.946047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:59.992059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:00.017858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:00.040946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c37bf-97fa-46bd-8def-98008f6b379f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.065976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.065976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.065976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.065976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] PATH_INFO: `/v3/projects/5714394086f24b9fa6a155436ece02a6` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.066789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5714394086f24b9fa6a155436ece02a6)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.066789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.085165 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 816/1639] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:00.094551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.127189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18735c6a-e344-4a62-8e2b-c9e2f53e280d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.136584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.137290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.137864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.138272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.139147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.141469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.344226 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 817/1640] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:00 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:00.358357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.362602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5714394086f24b9fa6a155436ece02a6', 'request_id': 'req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.369191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5714394086f24b9fa6a155436ece02a6', 'request_id': 'req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.376684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5714394086f24b9fa6a155436ece02a6', 'request_id': 'req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.438079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff86c6-1f1a-42c7-b83d-11a889805670 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.444247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.444878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.445680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.446080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.446840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.449821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3ea8e-5397-418b-a3d5-b06505a4cf0a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.491428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] The token cache is being invalidate because project 5714394086f24b9fa6a155436ece02a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:00.491779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.493892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.499609 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 824/1641] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:09:59 2022] DELETE /identity/v3/projects/5714394086f24b9fa6a155436ece02a6 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:00.499609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.502252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.571178 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 818/1642] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:00 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:00.571178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.571178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.571178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.585123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecf-d07c-4802-ae0b-fd26ec4554cc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.590962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.591283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.593795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.593795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] PATH_INFO: `/v3/users/d2d7dfdc3af24ec2ac27115540c20395` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.593795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d7dfdc3af24ec2ac27115540c20395)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.597064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.600681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:00.602447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.613459 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 819/1643] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:00.621212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.623423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.641965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714cc6e-c160-4b8a-80fa-79992f7ef226 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.646764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.647424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.647735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.648009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.648765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.650439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe697e40-dd25-416a-baeb-0ad20bd87d25 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.664734 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 820/1644] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:00.664734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.664734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.664734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.693521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:00.693521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.715773 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 821/1645] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:00.719276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.731742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703703c4-3b86-46ad-99d6-de6d7b7eda90 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.737123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.737881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.738832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.739183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] PATH_INFO: `/v3/projects/43ef99efbb494302916bcee855c7860b/users/b37dcb02b20648e6a02e85189c186ee6/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.759490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ef99efbb494302916bcee855c7860b, user_id=b37dcb02b20648e6a02e85189c186ee6, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.762380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.846803 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d7dfdc3af24ec2ac27115540c20395'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.847876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d7dfdc3af24ec2ac27115540c20395'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.862704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d7dfdc3af24ec2ac27115540c20395'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.873973 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 822/1646] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:00 2022] PUT /identity/v3/projects/43ef99efbb494302916bcee855c7860b/users/b37dcb02b20648e6a02e85189c186ee6/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:00.879479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.879479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.955460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f24beb-3fad-4f55-a00f-a1c65ac39bf4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:00.960496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:00.961639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:00.961922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:00.962225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:00.962911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:00.963296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:00.965050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:00.981188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 823/1647] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:00.982219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:00.983940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:00.993144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d7dfdc3af24ec2ac27115540c20395'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:00.998053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c6e8-a6b0-4b7d-b1f3-3c191277c146 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.002883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d7dfdc3af24ec2ac27115540c20395'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:01.003876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.004624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.004808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.005042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.005723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.007358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b9cb7-151c-4c6e-b141-56196d110a6b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.011529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 825/1648] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:00 2022] DELETE /identity/v3/users/d2d7dfdc3af24ec2ac27115540c20395 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:01.012205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.021842 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 824/1649] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:01.021842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.021842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.021842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.044813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:01.046062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:01.098287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc23f5d-3579-401e-a4a0-37426fc68b3c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.103741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.104475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.104933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.105263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.106272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.108461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8483365-f3d3-4b0a-ab87-6c2b885926a6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.123650 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 825/1650] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:01.123650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.132075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d555350-2ff1-4944-88e8-db86162ff449 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.139062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.140546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.140940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.141237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] PATH_INFO: `/v3/projects/43ef99efbb494302916bcee855c7860b/users/b37dcb02b20648e6a02e85189c186ee6/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.143745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ef99efbb494302916bcee855c7860b, user_id=b37dcb02b20648e6a02e85189c186ee6, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.146774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.160180 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 826/1651] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:01 2022] PUT /identity/v3/projects/43ef99efbb494302916bcee855c7860b/users/b37dcb02b20648e6a02e85189c186ee6/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:01.161034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.162667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:01.226067 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 826/1652] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:01 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:01.226944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f538eb-dade-4d99-a1bb-038b15145a11 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.227656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.228188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.228529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.261447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] MFA Rules not processed for user `b37dcb02b20648e6a02e85189c186ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:01.262967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:01.284538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.285152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.285436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.285713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.351997 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 827/1653] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:01.351997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.377859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.379541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.395786 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 827/1654] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:01.397422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.400142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:01.420112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:01.431509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd145d58-db63-4a63-9867-e37945447d92 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.437707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.438680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.438966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.439244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.439932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.441911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.476419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35dffe-7ebb-4a54-bc2b-6b06e184bc28 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.480050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:01.482079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.482718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.483004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.483283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.485206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:01.552555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.554026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.565064 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 828/1655] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:01.576522 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 828/1656] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:01 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:01.579613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.590292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc445c3-e26e-4937-b69d-45b0ce53c507 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.596250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.596931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.597213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.597539 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.598185 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.600658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.624030 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 829/1657] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:01.629507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.643429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace4e8b-b774-4df4-b784-c8b106b42e68 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.672658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.674047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.674866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.675351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.676059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.682754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f564f-cbad-4e34-92bb-76f61d8a6f0b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.708699 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 829/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:01.708943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:01.723834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4fcf-d795-4859-b65e-9bdf3f23d12f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:01.731543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:01.732276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.732564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.733478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] PATH_INFO: `/v3/projects/94956376a4dc4d80b651620998b1fb79/users/62ac4c19ac1e492ebf71e12657bb12f5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.760594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94956376a4dc4d80b651620998b1fb79, user_id=62ac4c19ac1e492ebf71e12657bb12f5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:01.763915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:01.904347 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 830/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:01 2022] PUT /identity/v3/projects/94956376a4dc4d80b651620998b1fb79/users/62ac4c19ac1e492ebf71e12657bb12f5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:01.920829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:01.921400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:01.921400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:01.979234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] MFA Rules not processed for user `62ac4c19ac1e492ebf71e12657bb12f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:01.980523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:02.084806 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 830/1660] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:02.088706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.173630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03b56-0124-4c03-80fa-795afbc86ed7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:02.180106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:02.180464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:02.180720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:02.181000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:02.181662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:02.183113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.184825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:02.196221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:02.206652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6436-71cf-4ef6-ba94-85056c29f936 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:02.222781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:02.223688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:02.224112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:02.224453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] PATH_INFO: `/v3/projects/7ee155a421704255933d003ea9d593a9` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:02.227110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee155a421704255933d003ea9d593a9)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:02.231763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:02.385294 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 831/1661] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:02 2022] POST /identity/v3/projects => generated 375 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:02.387808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.453819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee155a421704255933d003ea9d593a9', 'request_id': 'req-30a4b676-1d05-40b2-bdd9-22b68b07ce58'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:02.463896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee155a421704255933d003ea9d593a9', 'request_id': 'req-30a4b676-1d05-40b2-bdd9-22b68b07ce58'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:02.492749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee155a421704255933d003ea9d593a9', 'request_id': 'req-30a4b676-1d05-40b2-bdd9-22b68b07ce58'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:02.545090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:02.546379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef3984-19b6-4667-83ad-152b9e36ba34 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:02.586339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] The token cache is being invalidate because project 7ee155a421704255933d003ea9d593a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:02.588662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:02.588662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:02.592014 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 831/1662] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:02 2022] DELETE /identity/v3/projects/7ee155a421704255933d003ea9d593a9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:02.626063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.630191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:02.690248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:02.700057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:02.704449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:02.704449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:02.706206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:02.711986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:02.830422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b676-1d05-40b2-bdd9-22b68b07ce58 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:02.849631 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 832/1663] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:02 2022] POST /identity/v3/users => generated 294 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:02.853523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.857118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:02.858511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:02.859687 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:02.864155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:02.864782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] PATH_INFO: `/v3/users/bd0ade11b6b84c0e9cfdc9eb0025f966` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:02.868941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0ade11b6b84c0e9cfdc9eb0025f966)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:02.873998 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:02.891277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baffb6-f1f1-4351-82e6-51704c94e62a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:02.900131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:02.901037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:02.901691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:02.902197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:02.903117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:02.905778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:02.920477 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 833/1664] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:02.921824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:02.923968 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:03.003725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede57653-60ee-40ed-af23-a5a341a1bf44 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:03.007612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:03.008825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:03.009107 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:03.009381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:03.011283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:03.089041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:03.092632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:03.109951 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 834/1665] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:03.113819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:03.135432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a4f93-67e4-4a5f-9cdd-d138c3ff84cc service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:03.142490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:03.143111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:03.143358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:03.143694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] PATH_INFO: `/v3/projects/83cca0871b0249fa8f897bf27f03bb9b/users/72263c6595044c8f9e1234b41d6c4e34/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:03.168417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:03.173065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0ade11b6b84c0e9cfdc9eb0025f966'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:03.174480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0ade11b6b84c0e9cfdc9eb0025f966'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:03.184487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83cca0871b0249fa8f897bf27f03bb9b, user_id=72263c6595044c8f9e1234b41d6c4e34, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:03.192209 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:03.204477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0ade11b6b84c0e9cfdc9eb0025f966'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:03.286700 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 835/1666] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:03 2022] PUT /identity/v3/projects/83cca0871b0249fa8f897bf27f03bb9b/users/72263c6595044c8f9e1234b41d6c4e34/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:03.291423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:03.293536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:03.313631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0ade11b6b84c0e9cfdc9eb0025f966'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:03.326731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0ade11b6b84c0e9cfdc9eb0025f966'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:03.345075 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 832/1667] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:02 2022] DELETE /identity/v3/users/bd0ade11b6b84c0e9cfdc9eb0025f966 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:03.455814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020d465-f6db-4526-9669-9e06311076b7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:03.598014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:03.598659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:03.598943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:03.599222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:03.599959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:03.602449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5e42c-b49a-428b-8fd0-69572cd60cf6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:03.616136 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 836/1668] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:03.625927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:03.643351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc56db-7408-49e0-bb94-feab75c4775a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:03.650971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:03.651859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:03.652289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:03.652697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] PATH_INFO: `/v3/projects/83cca0871b0249fa8f897bf27f03bb9b/users/72263c6595044c8f9e1234b41d6c4e34/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:03.655092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83cca0871b0249fa8f897bf27f03bb9b, user_id=72263c6595044c8f9e1234b41d6c4e34, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:03.658377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:03.686324 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 833/1669] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:03 2022] PUT /identity/v3/projects/83cca0871b0249fa8f897bf27f03bb9b/users/72263c6595044c8f9e1234b41d6c4e34/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:03.703201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:03.703707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:03.704084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:03.813062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] MFA Rules not processed for user `72263c6595044c8f9e1234b41d6c4e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:03.815632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.031942 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 837/1670] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:04.051928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:04.055505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.173097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43b728-be17-43ec-9752-c43ad008131e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:04.179889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:04.181000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:04.181506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:04.182045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:04.185318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.265098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:04.267475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.354088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:04.357633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb297-2a3f-4398-9917-d07a03168124 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:04.378974 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 834/1671] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:04.419154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3621-2d67-41e7-9191-2fa73c266cd9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:04.429193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:04.430971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:04.431665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:04.432249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] PATH_INFO: `/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:04.434470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa81d657a5d4a94b1fcd5a505c2acf5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:04.438046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:04.492985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:04.589717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa81d657a5d4a94b1fcd5a505c2acf5', 'request_id': 'req-2f5a9444-4a71-4e17-bd21-aeab51b90051'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:04.596768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa81d657a5d4a94b1fcd5a505c2acf5', 'request_id': 'req-2f5a9444-4a71-4e17-bd21-aeab51b90051'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:04.687303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa81d657a5d4a94b1fcd5a505c2acf5', 'request_id': 'req-2f5a9444-4a71-4e17-bd21-aeab51b90051'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:04.714667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] The token cache is being invalidate because project eaa81d657a5d4a94b1fcd5a505c2acf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:04.715111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:04.715961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2f5a9444-4a71-4e17-bd21-aeab51b90051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:04.717716 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 838/1672] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:04 2022] DELETE /identity/v3/projects/eaa81d657a5d4a94b1fcd5a505c2acf5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:04.723505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e59b95-2830-488e-a821-383fd72c288c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:04.723818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e59b95-2830-488e-a821-383fd72c288c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:04.724096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e59b95-2830-488e-a821-383fd72c288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:04.755554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-69e59b95-2830-488e-a821-383fd72c288c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:04.758274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-69e59b95-2830-488e-a821-383fd72c288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.907525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 835/1673] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:04.915687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:04.915998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:04.916253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:04.942489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:04.944020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:04.963286 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 839/1674] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:04.964418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e59b95-2830-488e-a821-383fd72c288c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:04.989917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e59b95-2830-488e-a821-383fd72c288c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:05.004418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:05.005367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:05.005811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:05.006282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:05.007233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:05.007793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:05.011075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:05.027062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:05.050008 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 836/1675] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:05.052381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:05.071735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2993ef-3732-4823-a81e-5a9dd2e9a62c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:05.092684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:05.094098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:05.094605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:05.095114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:05.096446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:05.100864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:05.284465 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 840/1676] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:05 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:05.287405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:05.328564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec50d7-0d6c-42d4-b75a-c57e1e50a3c8 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:05.559361 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:05.560625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:05.561120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:05.561594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:05.562761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:05.567919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:05.706347 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 837/1677] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:05 2022] POST /identity/v3/users => generated 288 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:05.713715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:05.751236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037da0-f7d3-414c-8114-69237a3f2483 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:05.782046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:05.783117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:05.783761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:05.784550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:05.785906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:05.789685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:05.833500 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 841/1678] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:05.841980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:05.886564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f2b7-7b49-4d18-9a19-af6459013fe6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:05.908125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:05.911927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:05.912739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:05.913724 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:05.916332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:05.918291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31a4a4-f5c1-442a-acee-b33883ea5887 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:05.960222 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 838/1679] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:05.967118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:05.992708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b0ee-7bea-4c2a-9d5d-f81598531508 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:06.005224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:06.006657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:06.008109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:06.010476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] PATH_INFO: `/v3/projects/91d84d0f5a664ad9bd037b414be0425a/users/07b5f71883e94de98f54c07a41f15497/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:06.046542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d84d0f5a664ad9bd037b414be0425a, user_id=07b5f71883e94de98f54c07a41f15497, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:06.056236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:06.184721 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 842/1680] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:05 2022] PUT /identity/v3/projects/91d84d0f5a664ad9bd037b414be0425a/users/07b5f71883e94de98f54c07a41f15497/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:06.194315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:06.205235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:06.205235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:06.291408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] MFA Rules not processed for user `07b5f71883e94de98f54c07a41f15497`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:06.292310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:06.329118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:06.333627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:06.467849 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 839/1681] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:06.467849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:06.482110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b7451-cf23-484d-b76b-d5d0d0bc5ea5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:06.507485 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:06.514272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:06.514272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:06.514272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:06.518562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:06.581492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:06.644206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10e98-2e24-4868-aebc-b64c7a13127a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:06.658770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:06.661427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:06.664432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:06.664432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:06.665348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:06.670360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:06.673882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:06.677053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:06.716744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:06.722123 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 843/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:06.846972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:06.848274 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 840/1683] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:06 2022] POST /identity/v3/projects => generated 375 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:06.866416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b20f-f8d0-449d-84ae-47e7d15f262c service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:06.959016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:06.959689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:06.960118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:06.960268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:06.961420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:06.963400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.057704 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 844/1684] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:06 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:07.059389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:07.076983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de4085-16ca-4083-a85b-cebff39009f1 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:07.083353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:07.084146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.084438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.084721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.086675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:07.088486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.103235 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 841/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:07.104914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:07.124919 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d087-817f-444d-9730-dd4c1bfd705e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:07.134402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:07.135462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.136003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.136507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] PATH_INFO: `/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f/users/01c5db086b044ef2a8220913f7a248a4/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.162428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05fa29f0e2b46fd9d0279a9ddda372f, user_id=01c5db086b044ef2a8220913f7a248a4, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:07.168787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.289074 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 845/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:07 2022] PUT /identity/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f/users/01c5db086b044ef2a8220913f7a248a4/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:07.294334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:07.306592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e704ce-4402-4593-8daf-7803a1870147 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:07.379466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:07.380268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.380669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.381182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.381944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:07.384230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328cd12-4097-41eb-a19f-dbccd5240f13 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.401858 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 842/1687] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:07.403037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:07.414698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663c5d-aa6e-48b1-bcda-a90ceefbb317 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:07.437056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:07.438288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.438784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.439400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] PATH_INFO: `/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f/users/01c5db086b044ef2a8220913f7a248a4/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.441974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05fa29f0e2b46fd9d0279a9ddda372f, user_id=01c5db086b044ef2a8220913f7a248a4, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:07.446119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.465942 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 846/1688] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:07 2022] PUT /identity/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f/users/01c5db086b044ef2a8220913f7a248a4/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:07.472188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.472592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.473080 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.511911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] MFA Rules not processed for user `01c5db086b044ef2a8220913f7a248a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:07.513191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:07.538008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:07.590739 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 843/1689] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:07.741326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:07.743907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:07.813625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf187f3-2cfb-44d0-83cb-adbdf4dd0f39 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:07.821602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:07.822236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:07.822537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:07.822813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:07.889789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:07.891312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:07.903273 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 847/1690] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:08.700189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:08.704129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:08.793732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:08.881694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac1c22-a528-4de5-937a-4660bee6589c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:08.891762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:08.892418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:08.892777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:08.893164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] PATH_INFO: `/v3/users/62ac4c19ac1e492ebf71e12657bb12f5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:08.894487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ac4c19ac1e492ebf71e12657bb12f5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:08.896631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:09.084920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:09.099679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ace661-bce1-4b51-8e0e-a843714410bc service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:09.106434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:09.107498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:09.108082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:09.108726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:09.111850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:09.194373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ac4c19ac1e492ebf71e12657bb12f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:09.195557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ac4c19ac1e492ebf71e12657bb12f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:09.219641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ac4c19ac1e492ebf71e12657bb12f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:09.246883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:09.249206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c15dd-b125-4da4-a9db-ce6826571862 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:09.261032 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 848/1691] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:09.307092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ac4c19ac1e492ebf71e12657bb12f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:09.315202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ac4c19ac1e492ebf71e12657bb12f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:09.322359 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 844/1692] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:08 2022] DELETE /identity/v3/users/62ac4c19ac1e492ebf71e12657bb12f5 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:09.326810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:09.327149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:09.327440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:09.356957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:09.359231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:09.440942 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 849/1693] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:09.451638 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:09.453082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:09.458086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:09.519537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea9a3-7c57-4cdd-aa17-48c39fe30c75 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:09.523978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:09.524594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:09.524878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:09.525158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:09.535831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:09.537223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:09.549929 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 845/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:10.175332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:10.176941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:10.191606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec25ef2-ee98-4548-a6cd-b34f7b291bb2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:10.198345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:10.199148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:10.199459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:10.199799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] PATH_INFO: `/v3/projects/94956376a4dc4d80b651620998b1fb79` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:10.210607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94956376a4dc4d80b651620998b1fb79)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:10.210607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:10.418153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94956376a4dc4d80b651620998b1fb79', 'request_id': 'req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.431503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94956376a4dc4d80b651620998b1fb79', 'request_id': 'req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.468704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:10.520393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94956376a4dc4d80b651620998b1fb79', 'request_id': 'req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.538632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] The token cache is being invalidate because project 94956376a4dc4d80b651620998b1fb79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:10.538940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.539676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.541368 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 850/1695] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:10 2022] DELETE /identity/v3/projects/94956376a4dc4d80b651620998b1fb79 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:10.545982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:10.558318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b990c-1ac2-4ca1-81e3-45a738c5a4e6 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:10.654455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:10.657017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:10.657017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:10.657017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] PATH_INFO: `/v3/users/72263c6595044c8f9e1234b41d6c4e34` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:10.658437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72263c6595044c8f9e1234b41d6c4e34)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:10.662024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:10.862072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72263c6595044c8f9e1234b41d6c4e34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.862789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72263c6595044c8f9e1234b41d6c4e34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.876975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72263c6595044c8f9e1234b41d6c4e34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.936239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72263c6595044c8f9e1234b41d6c4e34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.945755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72263c6595044c8f9e1234b41d6c4e34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:10.952892 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 846/1696] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:10 2022] DELETE /identity/v3/users/72263c6595044c8f9e1234b41d6c4e34 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:10.974314 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:11.050910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:11.053941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:11.076477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac229-a5dd-4bb3-8c90-7d7db385b9e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:11.153605 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:11.154260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.154542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.154818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] PATH_INFO: `/v3/users/47b44e4d5a5344e2938d7551626ab287` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:11.156061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b44e4d5a5344e2938d7551626ab287)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:11.158018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:11.458354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b44e4d5a5344e2938d7551626ab287'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.460705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b44e4d5a5344e2938d7551626ab287'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.481213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b44e4d5a5344e2938d7551626ab287'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.484484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b44e4d5a5344e2938d7551626ab287'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.509864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-565c2005-8617-468a-b4be-3f407c26c77e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b44e4d5a5344e2938d7551626ab287'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.569632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:11.569632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:11.600107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b861cb-71af-4069-a11c-76c03efe24b0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:11.610056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:11.621969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.622377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.622665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] PATH_INFO: `/v3/projects/83cca0871b0249fa8f897bf27f03bb9b` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:11.629372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83cca0871b0249fa8f897bf27f03bb9b)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:11.635275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:11.653019 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 851/1697] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:11 2022] DELETE /identity/v3/users/47b44e4d5a5344e2938d7551626ab287 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:11.659735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.660364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.680380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:11.727384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:11.728692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:11.740147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:11.808895 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 852/1698] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:11.823497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:11.824998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:11.898468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0a49-cffa-4c56-90fc-5b79a335b820 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:11.904763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83cca0871b0249fa8f897bf27f03bb9b', 'request_id': 'req-ebfd7862-2b90-426a-b605-4653a5f34a8a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.915159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83cca0871b0249fa8f897bf27f03bb9b', 'request_id': 'req-ebfd7862-2b90-426a-b605-4653a5f34a8a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.926675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83cca0871b0249fa8f897bf27f03bb9b', 'request_id': 'req-ebfd7862-2b90-426a-b605-4653a5f34a8a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:11.958932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:11.959574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.959920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.961976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.055494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.057223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017faa4d-de99-4579-802a-d0a7d0b9817e service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.070654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] The token cache is being invalidate because project 83cca0871b0249fa8f897bf27f03bb9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:12.070928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:12.072738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ebfd7862-2b90-426a-b605-4653a5f34a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:12.074555 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 847/1699] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:11 2022] DELETE /identity/v3/projects/83cca0871b0249fa8f897bf27f03bb9b => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:12.076288 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 853/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:12.087619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.088115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.089239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.128913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:12.131495 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:12.150201 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 848/1701] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:12.158876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.159344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.160093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.188200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:12.189757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:12.206383 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 854/1702] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:12.207200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.219572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd87fa-0b70-403b-a329-860b370ae980 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.225698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.228982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.229185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.229519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.230457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:12.230844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.233011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.249680 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 849/1703] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:12.252189 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.264035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b478fc2-c2cb-49e4-9c4a-938c41b0a811 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.271084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.273350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.273856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.274352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.275541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.279078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.287143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:12.428864 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 855/1704] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:12 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:12.435068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.452089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60a197-f6a6-4a93-90da-3f32c31a0e3c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.500675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:12.557698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.561658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:12.581272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.582057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.582251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.582947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.583987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.587783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.704898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f640b4-5ee9-4a3e-820d-ff7f1676c7b4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.711046 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 850/1705] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:12 2022] POST /identity/v3/users => generated 292 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:12.715214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.715749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:12.721568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.722675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.723178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.723768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.727022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:12.740308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:12.753379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dfe7a-534f-4a90-b922-d5d1ca24da77 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.761886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.763515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.764037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.765251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] PATH_INFO: `/v3/users/7917e9f7c9b544b8bf17c66da1f949a8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.767021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7917e9f7c9b544b8bf17c66da1f949a8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.769382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.823239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:12.896326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.897920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.910378 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 856/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:12.911258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.930844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e83283-86e5-484f-b861-6cbebe350f8e service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.939187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:12.940259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:12.940732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:12.941287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:12.943967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:12.950460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:12.964424 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 857/1707] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:12.971266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:12.994545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7917e9f7c9b544b8bf17c66da1f949a8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:12.996081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d70c-de01-4625-a680-a4ec04b452e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:12.996948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7917e9f7c9b544b8bf17c66da1f949a8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.017075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.017867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.018238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.019171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.022533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.025930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.031085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7917e9f7c9b544b8bf17c66da1f949a8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.054577 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 858/1708] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:13.061936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:13.064827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.087012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f174ef-e275-40be-8da1-c649a329f658 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:13.093863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.094582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.094867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.095154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] PATH_INFO: `/v3/projects/6c2eb5277d964cd1a8383e3fb8565a62/users/21f1bc0913d5468eb8ebad7dc2573b56/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.115207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2eb5277d964cd1a8383e3fb8565a62, user_id=21f1bc0913d5468eb8ebad7dc2573b56, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.121547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641c7d2-5443-4c6a-ac28-d1a89f75d591 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.138469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7917e9f7c9b544b8bf17c66da1f949a8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.157881 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7917e9f7c9b544b8bf17c66da1f949a8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.170744 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 851/1709] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:12 2022] DELETE /identity/v3/users/7917e9f7c9b544b8bf17c66da1f949a8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:13.181945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:13.190335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.227728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:13.298769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95a5c9-b151-4d53-91f8-3fab55677473 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:13.308885 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 859/1710] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:13 2022] PUT /identity/v3/projects/6c2eb5277d964cd1a8383e3fb8565a62/users/21f1bc0913d5468eb8ebad7dc2573b56/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:13.311258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39589-6d6d-4451-8eef-0fcf066c4a24 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.311803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39589-6d6d-4451-8eef-0fcf066c4a24 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.312604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39589-6d6d-4451-8eef-0fcf066c4a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.340624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-77a39589-6d6d-4451-8eef-0fcf066c4a24 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:13.342360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-77a39589-6d6d-4451-8eef-0fcf066c4a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.405099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.406170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.406625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.407142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] PATH_INFO: `/v3/projects/aa4bdec6d03b414882da5ca42b0a2dea` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.408693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4bdec6d03b414882da5ca42b0a2dea)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.411239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.423484 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 860/1711] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:13.423484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.423484 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.424514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.458440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] MFA Rules not processed for user `21f1bc0913d5468eb8ebad7dc2573b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:13.459759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.573832 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 861/1712] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:13.576451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:13.580121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.653997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4bdec6d03b414882da5ca42b0a2dea', 'request_id': 'req-43b2f77d-6802-4637-92e4-babb852906e9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.664722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4bdec6d03b414882da5ca42b0a2dea', 'request_id': 'req-43b2f77d-6802-4637-92e4-babb852906e9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.667431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7ca5e-83cd-4f28-8be2-68feb005db39 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:13.673277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4bdec6d03b414882da5ca42b0a2dea', 'request_id': 'req-43b2f77d-6802-4637-92e4-babb852906e9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.706057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:13.742742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.743330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.743642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.743920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.800473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] The token cache is being invalidate because project aa4bdec6d03b414882da5ca42b0a2dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:13.801004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.801756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:13.807844 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 852/1713] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:13 2022] DELETE /identity/v3/projects/aa4bdec6d03b414882da5ca42b0a2dea => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:13.808843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:13.810320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:13.857528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.859014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.870281 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 862/1714] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:13.871143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:13.885145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66d4e-fbeb-4ccc-89d6-99421bcde483 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:13.890481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.891138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.891440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.891727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] PATH_INFO: `/v3/users/5aeebbd08b62488ab0aa5334408942f6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.893143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aeebbd08b62488ab0aa5334408942f6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.895684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.918959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2f77d-6802-4637-92e4-babb852906e9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:13.928690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:13.932123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:13.932573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:13.933110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:13.934264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:13.938758 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:13.943871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:14.097210 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 853/1715] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:13 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:14.103294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:14.175769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aeebbd08b62488ab0aa5334408942f6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.176816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aeebbd08b62488ab0aa5334408942f6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.205568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aeebbd08b62488ab0aa5334408942f6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.207361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aeebbd08b62488ab0aa5334408942f6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.260238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687a18-9558-42e5-9433-216dc49d90cd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:14.262523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aeebbd08b62488ab0aa5334408942f6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.266438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:14.271366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.271845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.273121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.273987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:14.276209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:14.378858 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 863/1716] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:13 2022] DELETE /identity/v3/users/5aeebbd08b62488ab0aa5334408942f6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:14.397041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:14.410587 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 854/1717] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:14 2022] POST /identity/v3/users => generated 292 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:14.412145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:14.415719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:14.551093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff1f8-ffdd-4cd8-83b9-a1334cd512b0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:14.560686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:14.561609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:14.561786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.562532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.563028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.566738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:14.567832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:14.583016 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 864/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:14.592448 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:14.604138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee263b-7fea-4b72-a1f1-9c6372da08d5 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:14.609706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:14.610378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.610664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.610944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.611647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:14.613318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:14.631817 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 855/1719] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:14.636910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:14.637320 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:14.640136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:14.661494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6149e-0e64-440d-a657-85171e78fefa admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:14.668150 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e4e5-c279-4686-807d-afd16002fb8f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:14.669225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:14.670490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.671070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.671975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] PATH_INFO: `/v3/projects/0fff6034ae6949139034d9d30e386166/users/0c9bac418f414127949146e54bdbfac0/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.691742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:14.691742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.691742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.691742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] PATH_INFO: `/v3/projects/bba669eb2db44f6eb7c3f2fdb419c5c3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.692089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba669eb2db44f6eb7c3f2fdb419c5c3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:14.693779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:14.706134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fff6034ae6949139034d9d30e386166, user_id=0c9bac418f414127949146e54bdbfac0, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:14.712253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0202c2-a545-4eed-8319-ca994ceb8256 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:14.803721 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 856/1720] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:14 2022] PUT /identity/v3/projects/0fff6034ae6949139034d9d30e386166/users/0c9bac418f414127949146e54bdbfac0/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:14.816003 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30509-d12b-4f64-a797-b110953519ae None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:14.816318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30509-d12b-4f64-a797-b110953519ae None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:14.816633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30509-d12b-4f64-a797-b110953519ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:14.852158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-72a30509-d12b-4f64-a797-b110953519ae None None] MFA Rules not processed for user `0c9bac418f414127949146e54bdbfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:14.853400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-72a30509-d12b-4f64-a797-b110953519ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:14.944299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba669eb2db44f6eb7c3f2fdb419c5c3', 'request_id': 'req-3bbc8e27-d133-46cb-9747-aeae1c068b7b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.962080 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 857/1721] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:14.963229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba669eb2db44f6eb7c3f2fdb419c5c3', 'request_id': 'req-3bbc8e27-d133-46cb-9747-aeae1c068b7b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:14.963412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a30509-d12b-4f64-a797-b110953519ae None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.054175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a30509-d12b-4f64-a797-b110953519ae None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.068426 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba669eb2db44f6eb7c3f2fdb419c5c3', 'request_id': 'req-3bbc8e27-d133-46cb-9747-aeae1c068b7b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.087561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] The token cache is being invalidate because project bba669eb2db44f6eb7c3f2fdb419c5c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:15.088092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.089139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3bbc8e27-d133-46cb-9747-aeae1c068b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.091446 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 865/1722] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:14 2022] DELETE /identity/v3/projects/bba669eb2db44f6eb7c3f2fdb419c5c3 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:15.096845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.097323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.098465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.123265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:15.125228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.171299 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.174096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.176107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.178267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.182368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.189755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f230e-e0c2-464a-a871-e902051ce124 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.235550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:15.261690 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 866/1723] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:15.263826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.265398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.286454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaa312-9bd3-49b6-81bb-ca28f453ec2d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.305187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.306034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.306365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.306746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] PATH_INFO: `/v3/projects/1e38e7d52fbe45f4a21bfc65fabd07bb` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.308467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e38e7d52fbe45f4a21bfc65fabd07bb)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.311018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.584985 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 858/1724] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:14 2022] POST /identity/v3/projects => generated 383 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:15.584985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.584985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.584985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.604462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:15.622224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e38e7d52fbe45f4a21bfc65fabd07bb', 'request_id': 'req-ab21bea1-b565-4d66-9e59-3e3b0f72d161'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.635403 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:15.637643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.640171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e38e7d52fbe45f4a21bfc65fabd07bb', 'request_id': 'req-ab21bea1-b565-4d66-9e59-3e3b0f72d161'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.758651 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 859/1725] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:15.761110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.763443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.769012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e38e7d52fbe45f4a21bfc65fabd07bb', 'request_id': 'req-ab21bea1-b565-4d66-9e59-3e3b0f72d161'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.782099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb2ced-a6a8-4432-837d-5e801364effa None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.788486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.789618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.790082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.790532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.791366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.794734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.796167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] The token cache is being invalidate because project 1e38e7d52fbe45f4a21bfc65fabd07bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:15.797046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.798463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:15.807723 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 867/1726] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:15 2022] DELETE /identity/v3/projects/1e38e7d52fbe45f4a21bfc65fabd07bb => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:15.809499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.813527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.837075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bea1-b565-4d66-9e59-3e3b0f72d161 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.844810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.845996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.846553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.847147 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.848707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:15.849313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.853413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.883065 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 868/1727] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:15.883065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.886019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.910136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5e7c6-2685-44e1-b74b-1b85e92871ec admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.919701 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 860/1728] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:15 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:15.922104 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.923456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.924714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.925089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.925565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] PATH_INFO: `/v3/users/4ebcbcaaef75444094b1753657ce7e0a` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.927704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebcbcaaef75444094b1753657ce7e0a)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.932403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.941429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3eaa-3293-4f71-bc19-9e7ef7d85550 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.950444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.951481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.952060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.952663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.953967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.958084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.996127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:16.118435 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 861/1729] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:15 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:16.118435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.118435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:16.215122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d790e2-46c8-48f2-be19-5d6a51090cdf admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.222414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.223413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.223763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.224238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.226336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.228018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebcbcaaef75444094b1753657ce7e0a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:16.229049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebcbcaaef75444094b1753657ce7e0a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:16.229462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.251196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebcbcaaef75444094b1753657ce7e0a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:16.252538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebcbcaaef75444094b1753657ce7e0a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:16.259432 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 862/1730] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:16.261048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.264947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:16.275543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebcbcaaef75444094b1753657ce7e0a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:16.283792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbd53d-39b0-47ff-8369-97edbb67cbdd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.284878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:16.291026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.291883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.292293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.292740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.293506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.295883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.381689 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 869/1731] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:15 2022] DELETE /identity/v3/users/4ebcbcaaef75444094b1753657ce7e0a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:16.383505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.385164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:16.469976 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 863/1732] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:16 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:16.473447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.505133 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08894f79-47b5-4598-8e68-674ccb74221a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.522551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708791e-ec8c-41c2-93e7-430426b60937 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.533604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.534299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.534582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.534854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] PATH_INFO: `/v3/projects/0a53708dc975481f88349132d5eeac4c/users/db05ad6f184a40289f737ce797911ed8/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.568103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a53708dc975481f88349132d5eeac4c, user_id=db05ad6f184a40289f737ce797911ed8, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.572047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.637878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.638696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.638978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.639257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.641021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.642702 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.655207 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 864/1733] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:16.659635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.671008 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 870/1734] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:16 2022] PUT /identity/v3/projects/0a53708dc975481f88349132d5eeac4c/users/db05ad6f184a40289f737ce797911ed8/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:16.675644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.683239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e8e9-cd6e-4192-a0a2-a440d61f0331 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.828934 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594fbc4-d30a-4aad-b097-162c38160cd0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.839706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.841051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.841826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.842856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.843371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.844525 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.845081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.845837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.846294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.848140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.848821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.853222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.875658 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 871/1735] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:16.878134 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 865/1736] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:16.881110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.887125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:16.906584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fcb6-2f65-4a5f-9b9c-8c363ac345aa admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.910838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f741683-97ad-4383-8ffe-27479823026a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:16.916529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.917676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.918408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.919009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] PATH_INFO: `/v3/projects/81af87ee47c1443e94dc76ff4f453c57/users/c6ed0abc3f524e438fb5ae26ecadaad1/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.923193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:16.924676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:16.929671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:16.929671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] PATH_INFO: `/v3/projects/0a53708dc975481f88349132d5eeac4c/users/db05ad6f184a40289f737ce797911ed8/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:16.931425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a53708dc975481f88349132d5eeac4c, user_id=db05ad6f184a40289f737ce797911ed8, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.939293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0c47b-7c98-4bd7-a0be-ab3bd6942fd1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.951544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81af87ee47c1443e94dc76ff4f453c57, user_id=c6ed0abc3f524e438fb5ae26ecadaad1, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:16.959917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f406f76-5597-45d1-a77a-d79d3d338135 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:16.986696 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 872/1737] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:16 2022] PUT /identity/v3/projects/0a53708dc975481f88349132d5eeac4c/users/db05ad6f184a40289f737ce797911ed8/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:17.002520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.002992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.003449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.049721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] MFA Rules not processed for user `db05ad6f184a40289f737ce797911ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:17.050747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.075365 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 866/1738] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:16 2022] PUT /identity/v3/projects/81af87ee47c1443e94dc76ff4f453c57/users/c6ed0abc3f524e438fb5ae26ecadaad1/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:17.086178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.086519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.086814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.126615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] MFA Rules not processed for user `c6ed0abc3f524e438fb5ae26ecadaad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:17.127926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.185883 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 873/1739] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:17.186097 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:17.199175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.274248 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 867/1740] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:17.277833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:17.285156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.354058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e4b6b-f250-445d-be7b-278f64503ad9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:17.426696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84598101-8a3b-48aa-ab9c-b52e7257ce2d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:17.433364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:17.433997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.434281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.434555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.438654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.473697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:17.474349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.474662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.474901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] PATH_INFO: `/v3/users/0e881f186b354b2585afdcc355088348` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.480081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e881f186b354b2585afdcc355088348)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:17.486274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:17.589849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:17.591641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:17.608581 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 868/1741] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:17.675168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:17.677240 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.696889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6e097-461d-4671-85ae-b74abfad88a1 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:17.701613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:17.702309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.702599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.714108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.723176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e881f186b354b2585afdcc355088348'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:17.724731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e881f186b354b2585afdcc355088348'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:17.733381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e881f186b354b2585afdcc355088348'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:17.734487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e881f186b354b2585afdcc355088348'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:17.739362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:17.740961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:17.767833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1e23f202-bc74-409a-a35c-839c88cf51cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e881f186b354b2585afdcc355088348'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:17.774792 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 869/1742] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:17.798019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:17.804406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:17.807310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:17.827634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793956d-6036-4f3b-82ec-46c3bab52d92 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:17.834686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:17.836333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.836532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.836847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] PATH_INFO: `/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.838544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3004db3ba714f7f8bac1a8bfc511daf)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:17.842579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:17.887667 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 874/1743] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:17 2022] DELETE /identity/v3/users/0e881f186b354b2585afdcc355088348 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:17.892424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:17.892748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:17.893060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:17.924837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:17.927061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:18.012009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3004db3ba714f7f8bac1a8bfc511daf', 'request_id': 'req-f8cd94ac-25b9-44a5-9908-88d440470226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:18.021946 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 875/1744] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:18.023106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3004db3ba714f7f8bac1a8bfc511daf', 'request_id': 'req-f8cd94ac-25b9-44a5-9908-88d440470226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:18.037400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:18.039987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3004db3ba714f7f8bac1a8bfc511daf', 'request_id': 'req-f8cd94ac-25b9-44a5-9908-88d440470226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:18.045569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:18.092932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:18.124490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1b676-daae-4d49-ac98-79586cd56519 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:18.177196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] The token cache is being invalidate because project e3004db3ba714f7f8bac1a8bfc511daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:18.177196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:18.177196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f8cd94ac-25b9-44a5-9908-88d440470226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:18.178019 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 870/1745] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:17 2022] DELETE /identity/v3/projects/e3004db3ba714f7f8bac1a8bfc511daf => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:18.187829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:18.188333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:18.188943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:18.197205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:18.198143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:18.198569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:18.198996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:18.201301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:18.232498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:18.232498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:18.300922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:18.302923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a23658-7703-421a-9ca9-dbc20dd7dbee service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:18.320400 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 876/1746] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:18.331794 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 871/1747] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:18.342452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:18.342626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:18.342799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:18.371409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:18.372740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:18.386636 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 877/1748] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:18.388324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:18.435697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182b225-019e-40c8-abce-adf3a1d6fb56 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:18.445809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:18.447826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:18.448548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:18.449876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:18.452064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:18.463968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:18.468215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:18.505525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 872/1749] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:18.513598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:18.534167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a474e-b5f5-4464-9c3e-40e9ab12c680 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:18.542851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:18.543769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:18.544274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:18.544763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:18.545967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:18.550291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:18.721883 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 878/1750] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:18 2022] POST /identity/v3/projects => generated 405 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:18.727894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:18.751575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49489-4c56-4a1c-a921-ca80fa230d4e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.040637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.041775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.042825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.044195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.048413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.052835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.240959 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 873/1751] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:18 2022] POST /identity/v3/users => generated 303 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:19.243792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.271765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfc349-6340-41bb-bcd3-cf6643440c8e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.284509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.292912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.292912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.292912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.292912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.292912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.315066 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 879/1752] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:19.316970 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.333215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.341591 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:19.349703 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcc65-93cc-4131-8c48-f099184d4660 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.369167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.369416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489eace-2022-408a-b100-2d7fbccdb1af admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.370306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.370775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.371223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.372114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.375550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.377683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.378488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.378862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.379367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] PATH_INFO: `/v3/projects/3fd7c367b3b84e8583949ed69bbaadee` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.380831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd7c367b3b84e8583949ed69bbaadee)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.383012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.396346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:19.405658 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 874/1753] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:19.406975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.419890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00517-2945-4f0f-abc4-59e65b1bad14 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.425546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.426418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.426735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.427068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] PATH_INFO: `/v3/projects/11a218c85d0c4750963a4c6e1bde2259/users/f4ea4013221345c8b133f07ac0bb15f5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.447321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a218c85d0c4750963a4c6e1bde2259, user_id=f4ea4013221345c8b133f07ac0bb15f5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.450653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72ec29-99fb-4352-adcb-5afccc82fb85 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.580016 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 875/1754] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:19 2022] PUT /identity/v3/projects/11a218c85d0c4750963a4c6e1bde2259/users/f4ea4013221345c8b133f07ac0bb15f5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:19.593626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.593893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.594180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.598062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd7c367b3b84e8583949ed69bbaadee', 'request_id': 'req-7165d751-48c0-4739-87f5-2e13b70e6d0d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.608819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd7c367b3b84e8583949ed69bbaadee', 'request_id': 'req-7165d751-48c0-4739-87f5-2e13b70e6d0d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.630260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] MFA Rules not processed for user `f4ea4013221345c8b133f07ac0bb15f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:19.631697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:19.729883 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 876/1755] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:19 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:19.729883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.760647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd7c367b3b84e8583949ed69bbaadee', 'request_id': 'req-7165d751-48c0-4739-87f5-2e13b70e6d0d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.807769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] The token cache is being invalidate because project 3fd7c367b3b84e8583949ed69bbaadee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:19.808792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.811093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.814008 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 880/1756] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:19 2022] DELETE /identity/v3/projects/3fd7c367b3b84e8583949ed69bbaadee => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:19.821318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.823693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:19.834440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573dde6-8f32-4001-b2a5-cf1021b2d588 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.844489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.845478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.845949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.846422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.847297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.852008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.858943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165d751-48c0-4739-87f5-2e13b70e6d0d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.871512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.876098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.876731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.877963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] PATH_INFO: `/v3/users/0f126159c3e84635989970a49a4cf81a` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.879936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f126159c3e84635989970a49a4cf81a)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.884997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.972330 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 877/1757] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:19 2022] POST /identity/v3/projects => generated 407 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:19.977030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.979011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:20.105911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc078448-2ed8-44b6-8166-b48a5610c8a2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.113608 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.114652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.115067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.115455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.116658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.119664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.128010 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:20.144775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f126159c3e84635989970a49a4cf81a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.145522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f126159c3e84635989970a49a4cf81a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.157079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f126159c3e84635989970a49a4cf81a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.158425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f126159c3e84635989970a49a4cf81a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.181736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f126159c3e84635989970a49a4cf81a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.320074 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 878/1758] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:19 2022] POST /identity/v3/users => generated 310 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:20.325506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:20.328134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:20.352500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a262fe-7f06-445d-b40a-271688d13b22 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.361451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.362325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.362713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.363139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.364913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.367846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.380467 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 881/1759] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:19 2022] DELETE /identity/v3/users/0f126159c3e84635989970a49a4cf81a => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:20.388311 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 879/1760] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:20.393460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:20.489621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489778-fd0a-4a97-98ed-1daebc97ec09 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.499061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.500472 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.500946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.501406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] PATH_INFO: `/v3/projects/c8d7a72076044e808f68794474fb58a3/users/5a460601771d4ab58631f215300d0cb4/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.532065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d7a72076044e808f68794474fb58a3, user_id=5a460601771d4ab58631f215300d0cb4, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.537411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.672954 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 882/1761] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:20 2022] PUT /identity/v3/projects/c8d7a72076044e808f68794474fb58a3/users/5a460601771d4ab58631f215300d0cb4/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:20.678714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:20.690007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde1b70-1019-40e9-b517-0ccc7137e95d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.789590 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.790671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.791056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.791437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.792478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.794079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dd900-9118-4848-8272-ea88dd14b9cc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.807903 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 880/1762] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:20.828402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:20.832403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ae1bc-d0a4-434d-8865-29c650f9f804 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.838364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.838989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.839281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.839619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] PATH_INFO: `/v3/projects/c8d7a72076044e808f68794474fb58a3/users/5a460601771d4ab58631f215300d0cb4/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.841973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d7a72076044e808f68794474fb58a3, user_id=5a460601771d4ab58631f215300d0cb4, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.844854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.871794 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 883/1763] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:20 2022] PUT /identity/v3/projects/c8d7a72076044e808f68794474fb58a3/users/5a460601771d4ab58631f215300d0cb4/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:20.880276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.884565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.884565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.914746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] MFA Rules not processed for user `5a460601771d4ab58631f215300d0cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:20.916043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:21.001998 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 881/1764] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:20 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:21.019956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:21.022558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:21.035353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:21.040823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:21.154453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd24d-989c-4180-8886-3994c221fe17 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:21.159533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:21.160367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:21.160649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:21.160932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:21.171942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:21.173568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:21.186686 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 884/1765] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:21 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:21.188935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b166a-6d35-43d7-8dbe-cebe5275ff5f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:21.197561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:21.198789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:21.199256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:21.199770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] PATH_INFO: `/v3/projects/cb5afdba437943dcaaf0018f0960f43f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:21.202027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5afdba437943dcaaf0018f0960f43f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:21.206199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:21.286187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:21.546894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5afdba437943dcaaf0018f0960f43f', 'request_id': 'req-906c1f04-bfd6-4135-aec7-81434f0f8e4a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:21.562173 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5afdba437943dcaaf0018f0960f43f', 'request_id': 'req-906c1f04-bfd6-4135-aec7-81434f0f8e4a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:21.585123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5afdba437943dcaaf0018f0960f43f', 'request_id': 'req-906c1f04-bfd6-4135-aec7-81434f0f8e4a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:21.705855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] The token cache is being invalidate because project cb5afdba437943dcaaf0018f0960f43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:21.706270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:21.707372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:21.709321 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 882/1766] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:21 2022] DELETE /identity/v3/projects/cb5afdba437943dcaaf0018f0960f43f => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:21.715756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:21.731726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17758-0a26-4427-8f7f-75c15328754c service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:21.813809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:21.814943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:21.815429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:21.815979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] PATH_INFO: `/v3/users/b37dcb02b20648e6a02e85189c186ee6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:21.818154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37dcb02b20648e6a02e85189c186ee6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:21.822301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:22.005554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:22.007053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:22.098071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1f04-bfd6-4135-aec7-81434f0f8e4a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:22.102802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:22.103807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:22.104287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:22.104774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:22.109010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:22.115656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37dcb02b20648e6a02e85189c186ee6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:22.117535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37dcb02b20648e6a02e85189c186ee6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:22.128469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37dcb02b20648e6a02e85189c186ee6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:22.129340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37dcb02b20648e6a02e85189c186ee6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:22.145884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37dcb02b20648e6a02e85189c186ee6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:22.189070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:22.190409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:22.201524 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 883/1767] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:22 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:22.268533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:22.279086 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 885/1768] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:21 2022] DELETE /identity/v3/users/b37dcb02b20648e6a02e85189c186ee6 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:22.462326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:22.464136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:22.476929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d156-c0e1-40f5-8306-6500f6b6b25a service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:22.565257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:22.566280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:22.566791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:22.567294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:22.571113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:22.693972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:22.695149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af5777-dd28-41b4-b55c-bfcb11419afb service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:22.706919 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 884/1769] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:22.915075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:22.985770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:22.989179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:23.086943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06195-f46f-48f3-85cf-eaeb7f56eddd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:23.092254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:23.092937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:23.093221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:23.093510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] PATH_INFO: `/v3/projects/43ef99efbb494302916bcee855c7860b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:23.094754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ef99efbb494302916bcee855c7860b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:23.096778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:23.269935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ef99efbb494302916bcee855c7860b', 'request_id': 'req-8ab6f3a5-6d09-4c16-9636-e145967d9189'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:23.277824 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ef99efbb494302916bcee855c7860b', 'request_id': 'req-8ab6f3a5-6d09-4c16-9636-e145967d9189'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:23.405917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ef99efbb494302916bcee855c7860b', 'request_id': 'req-8ab6f3a5-6d09-4c16-9636-e145967d9189'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:23.423371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] The token cache is being invalidate because project 43ef99efbb494302916bcee855c7860b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:23.423837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:23.425092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8ab6f3a5-6d09-4c16-9636-e145967d9189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:23.428340 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 886/1770] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:22 2022] DELETE /identity/v3/projects/43ef99efbb494302916bcee855c7860b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:23.434988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:23.436000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:23.436000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:23.463543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:23.464978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:23.541972 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 885/1771] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:23.550265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:23.550581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:23.550886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:23.583729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:23.585625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:23.594159 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 887/1772] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:23.598813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:23.612424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5e911-4aef-4201-8140-9aefe7c8ba0e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:23.620619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:23.621801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:23.622234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:23.622690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:23.624156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:23.624689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:23.627951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:23.652574 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 886/1773] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:23.661691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:23.685546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e005d59-a659-4b4f-8494-8b9ab97f1de0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:23.698253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:23.699239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:23.699863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:23.700504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:23.715535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:23.719387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:23.796188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:23.920338 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 888/1774] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:23 2022] POST /identity/v3/projects => generated 385 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:23.932400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:23.945626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7aaa9-4b84-47c0-a9d4-bee276aff0e7 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:24.046058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:24.046798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.047014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.047317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.048115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:24.050168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:24.098418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:24.166531 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 887/1775] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:23 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:24.172347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:24.192089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c1044-5288-4225-9e07-75bcb61ed7bc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:24.200911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:24.201816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.202284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.202757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.204412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:24.206370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:24.221178 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 889/1776] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:24.229381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:24.246700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef920f83-6759-4519-b480-9d2179d904d0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:24.253285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:24.254353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.254693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.255113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.256039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:24.258777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c7afe-195e-458c-b668-1fb16acc8b50 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:24.273063 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 888/1777] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:24.277573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:24.288615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2352-46ac-4502-bd62-779e9fbc4aef admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:24.294070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:24.294825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.295230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.295556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] PATH_INFO: `/v3/projects/13e170ac6f48451fbe5835640c187c88/users/c09236b3dce543c99818478b64533fcf/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.314193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e170ac6f48451fbe5835640c187c88, user_id=c09236b3dce543c99818478b64533fcf, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:24.316714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:24.433082 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 890/1778] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:24 2022] PUT /identity/v3/projects/13e170ac6f48451fbe5835640c187c88/users/c09236b3dce543c99818478b64533fcf/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:24.453282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.454122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.454293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.492596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] MFA Rules not processed for user `c09236b3dce543c99818478b64533fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:24.494952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:24.580572 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 889/1779] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:24.871863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:24.876065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:24.946080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d62ff-7c42-4569-a6b4-465b5237805e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:24.950364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:24.951126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:24.951410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:24.952061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:24.966964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:24.969738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:24.987665 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 891/1780] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:25.176833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:25.238106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:26.218389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:26.229854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214d1ef-23b6-4656-8dbd-0e74175c8469 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:26.236924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:26.237959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:26.239041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:26.240432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:26.242828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:26.364114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:26.367404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb0af3-77ac-4eae-a1f8-a0bb4ee5daa3 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:26.394542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:26.395527 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 890/1781] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:26.471625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:27.506420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:27.583393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:29.016089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:29.351497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:29.504857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ef6a-fe1b-402d-be6c-2ff997f28281 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:29.512512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:29.513655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:29.514256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:29.514715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] PATH_INFO: `/v3/users/c09236b3dce543c99818478b64533fcf` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:29.516899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09236b3dce543c99818478b64533fcf)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:29.521007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:29.720336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:29.818728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:29.844001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09236b3dce543c99818478b64533fcf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:29.871362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09236b3dce543c99818478b64533fcf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:29.876863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09236b3dce543c99818478b64533fcf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:29.878160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09236b3dce543c99818478b64533fcf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:29.903555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09236b3dce543c99818478b64533fcf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:30.038427 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 892/1782] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:29 2022] DELETE /identity/v3/users/c09236b3dce543c99818478b64533fcf => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:30.044491 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:30.045423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:30.046138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:30.101244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:30.102603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:30.322724 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 891/1783] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:30.347679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:30.353261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:30.513345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd38-bdb6-4422-9acd-9acc648e5eb2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:30.532475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:30.542805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:30.543285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:30.543871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:30.566994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:30.570941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1fac9-bbc7-476a-aa56-cff6bce360d5 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:30.597561 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 893/1784] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:30.798908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:30.798908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:30.833896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d19a2a-03d1-424d-8783-b2f7780efd34 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:30.851051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:30.852746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:30.852746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:30.853245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] PATH_INFO: `/v3/users/c6ed0abc3f524e438fb5ae26ecadaad1` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:30.855700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ed0abc3f524e438fb5ae26ecadaad1)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:30.861979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:31.176997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ed0abc3f524e438fb5ae26ecadaad1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.176997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ed0abc3f524e438fb5ae26ecadaad1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.200721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ed0abc3f524e438fb5ae26ecadaad1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.314433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ed0abc3f524e438fb5ae26ecadaad1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.340717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ed0abc3f524e438fb5ae26ecadaad1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.353407 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 892/1785] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:30 2022] DELETE /identity/v3/users/c6ed0abc3f524e438fb5ae26ecadaad1 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:31.360527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:31.362021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:31.362021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:31.396483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:31.398569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:31.486040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:31.487965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:31.526950 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 894/1786] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:31.535233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:31.539973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:31.561849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:31.577720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3964ff9-1c2a-4a21-b4bb-1a0f9bf4e611 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:31.592134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:31.593776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:31.594499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:31.595672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] PATH_INFO: `/v3/projects/13e170ac6f48451fbe5835640c187c88` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:31.601073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e170ac6f48451fbe5835640c187c88)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:31.611408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:31.698355 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf2d5-138a-47fb-b27c-44409532e1ad admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:31.704549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:31.705506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:31.705783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:31.706450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] PATH_INFO: `/v3/users/3e12208f0ceb41c8bb3b69cbd62e78b2` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:31.707978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e12208f0ceb41c8bb3b69cbd62e78b2)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:31.711692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:31.794567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e170ac6f48451fbe5835640c187c88', 'request_id': 'req-af7ddd0f-85df-48e0-acd5-53603233dfd0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.807369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e170ac6f48451fbe5835640c187c88', 'request_id': 'req-af7ddd0f-85df-48e0-acd5-53603233dfd0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.947457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e170ac6f48451fbe5835640c187c88', 'request_id': 'req-af7ddd0f-85df-48e0-acd5-53603233dfd0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.964629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] The token cache is being invalidate because project 13e170ac6f48451fbe5835640c187c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:31.965144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.966255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:31.969257 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 895/1787] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:31 2022] DELETE /identity/v3/projects/13e170ac6f48451fbe5835640c187c88 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:31.971361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:31.973759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:32.013039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e12208f0ceb41c8bb3b69cbd62e78b2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.016233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e12208f0ceb41c8bb3b69cbd62e78b2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.054835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e12208f0ceb41c8bb3b69cbd62e78b2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.093422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddd0f-85df-48e0-acd5-53603233dfd0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:32.099272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:32.099978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:32.100983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:32.101159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:32.103647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:32.172094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e12208f0ceb41c8bb3b69cbd62e78b2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.196723 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e12208f0ceb41c8bb3b69cbd62e78b2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.216106 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 893/1788] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:31 2022] DELETE /identity/v3/users/3e12208f0ceb41c8bb3b69cbd62e78b2 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:32.265214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:32.268534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:32.300295 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 896/1789] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:32.316456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:32.451049 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:32.452614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:32.530215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc9daa-202b-4690-8ffd-ac0360b69646 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:32.535255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:32.535953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:32.536235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:32.536510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] PATH_INFO: `/v3/users/f4ea4013221345c8b133f07ac0bb15f5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:32.538740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ea4013221345c8b133f07ac0bb15f5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:32.542175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:32.706454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:32.708860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:32.731105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbb56d-d3f4-466b-af9d-b268dbbf07c3 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:32.738967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:32.739902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:32.740293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:32.740678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] PATH_INFO: `/v3/users/21f1bc0913d5468eb8ebad7dc2573b56` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:32.742729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f1bc0913d5468eb8ebad7dc2573b56)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:32.746344 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:32.843452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ea4013221345c8b133f07ac0bb15f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.844486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ea4013221345c8b133f07ac0bb15f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.872395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ea4013221345c8b133f07ac0bb15f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:32.988831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ea4013221345c8b133f07ac0bb15f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.004711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ea4013221345c8b133f07ac0bb15f5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.023687 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 894/1790] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:32 2022] DELETE /identity/v3/users/f4ea4013221345c8b133f07ac0bb15f5 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:33.026560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:33.028837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.057916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f1bc0913d5468eb8ebad7dc2573b56'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.060073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f1bc0913d5468eb8ebad7dc2573b56'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.082839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f1bc0913d5468eb8ebad7dc2573b56'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.086405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f1bc0913d5468eb8ebad7dc2573b56'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.120502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9358a13f-5248-41ba-aa49-81408f9fa442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f1bc0913d5468eb8ebad7dc2573b56'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.176263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3e61-60e8-4eb4-bdd3-c6f550448564 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:33.181847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:33.182872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.183353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.183860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] PATH_INFO: `/v3/projects/445041c33bf944d8a7a39281e3aba6c5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.185607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=445041c33bf944d8a7a39281e3aba6c5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:33.187696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:33.243278 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 897/1791] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:32 2022] DELETE /identity/v3/users/21f1bc0913d5468eb8ebad7dc2573b56 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:33.246264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.246757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.247233 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.283393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:33.285156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.343675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445041c33bf944d8a7a39281e3aba6c5', 'request_id': 'req-b40b4979-b56d-4190-af54-bf5b1f3152ba'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.354153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445041c33bf944d8a7a39281e3aba6c5', 'request_id': 'req-b40b4979-b56d-4190-af54-bf5b1f3152ba'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.368879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445041c33bf944d8a7a39281e3aba6c5', 'request_id': 'req-b40b4979-b56d-4190-af54-bf5b1f3152ba'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.422926 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 898/1792] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:33.425082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:33.426747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.440014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352f3c-dc03-4202-b280-085775f07214 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:33.510025 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:33.510878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.511265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.511725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] PATH_INFO: `/v3/projects/81af87ee47c1443e94dc76ff4f453c57` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.513199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81af87ee47c1443e94dc76ff4f453c57)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:33.515399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:33.529816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] The token cache is being invalidate because project 445041c33bf944d8a7a39281e3aba6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:33.530297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.531606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b40b4979-b56d-4190-af54-bf5b1f3152ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.533734 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 895/1793] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:33 2022] DELETE /identity/v3/projects/445041c33bf944d8a7a39281e3aba6c5 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:33.536091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.536602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.537100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.556964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:33.567081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:33.568635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.578778 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 896/1794] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:33.579682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:33.581083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.660536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8aee4-d4c1-4ce1-911b-1f74b07e503e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:33.666259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:33.667172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.667615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.668610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.672289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.693478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:33.696224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279387c-a04f-4f3c-87dd-8f82d23d23cb service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:33.710124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81af87ee47c1443e94dc76ff4f453c57', 'request_id': 'req-317d53a3-4a56-4c1a-91f1-37964b8a0604'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.710997 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 897/1795] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:33.713419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13bb8c-a40a-4573-9431-252ba4eba29b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.713901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13bb8c-a40a-4573-9431-252ba4eba29b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.714382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13bb8c-a40a-4573-9431-252ba4eba29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.733699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81af87ee47c1443e94dc76ff4f453c57', 'request_id': 'req-317d53a3-4a56-4c1a-91f1-37964b8a0604'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.742960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2a13bb8c-a40a-4573-9431-252ba4eba29b None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:33.744916 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2a13bb8c-a40a-4573-9431-252ba4eba29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.798100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81af87ee47c1443e94dc76ff4f453c57', 'request_id': 'req-317d53a3-4a56-4c1a-91f1-37964b8a0604'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.824874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] The token cache is being invalidate because project 81af87ee47c1443e94dc76ff4f453c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:33.825754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.828002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:33.832519 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 899/1796] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:33 2022] DELETE /identity/v3/projects/81af87ee47c1443e94dc76ff4f453c57 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:33.834864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:33.839698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.890671 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 898/1797] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:33.893588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aaaf-56cf-4e7a-8751-3fb918a961b3 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.894348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aaaf-56cf-4e7a-8751-3fb918a961b3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.895021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aaaf-56cf-4e7a-8751-3fb918a961b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.941180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5b83aaaf-56cf-4e7a-8751-3fb918a961b3 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:33.945944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5b83aaaf-56cf-4e7a-8751-3fb918a961b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.953978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d53a3-4a56-4c1a-91f1-37964b8a0604 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:33.958861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:33.959059 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 899/1798] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:33.959981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.960559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.960985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.961696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081aef-f232-455d-b0fe-647c3d28923a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:33.962126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081aef-f232-455d-b0fe-647c3d28923a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:33.962762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081aef-f232-455d-b0fe-647c3d28923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:33.964166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:33.981459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:33.983352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:33.988918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5e081aef-f232-455d-b0fe-647c3d28923a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:33.990285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5e081aef-f232-455d-b0fe-647c3d28923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:34.002757 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 900/1799] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:34.005394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.005883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.006443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.017191 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 900/1800] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:34.019361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.039276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5890-c411-4f01-abac-7fae4987b448 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.053954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.054996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.055483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.055999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.056534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:34.076537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:34.076537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.076887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:34.077031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.077193 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 901/1801] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:34.077193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.089061 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 901/1802] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:34.096662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.097522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96305d3-cea3-4f86-8645-9e83d6d224ad None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.104405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.105543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.105938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.106226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.106979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:34.108037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.111542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.114749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8436f19-6ce9-4589-95c2-8d511f7b9c12 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.120947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.121738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.122118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.122636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.123871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.126419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.142256 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 902/1803] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:34.146518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.165056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767e65a-5aed-4028-b2a4-c5804890e99c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.176420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.177415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.177861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.179662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.180956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.187288 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.283524 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 902/1804] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:34 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:34.284879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.307061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 903/1805] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:34 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:34.314234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.418066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c503d-0a8e-4232-bf62-9d6aa675fba9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.432654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:34.449982 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580abbc-7806-45df-96d5-aaf2f828c521 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.460621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.463934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.463934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.463934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.463934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.468508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.520662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.521397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.521753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.522642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.523944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.525752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.604680 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 904/1806] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:34 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:34.606683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.610548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:34.633072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408e3b8-0f24-4587-adca-2dbb63caa844 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.636890 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 903/1807] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:34 2022] POST /identity/v3/users => generated 282 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:34.638319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:34.644062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.644370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:34.645524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.645798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.646302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] PATH_INFO: `/v3/users/07b5f71883e94de98f54c07a41f15497` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.649116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b5f71883e94de98f54c07a41f15497)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.653000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.660583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ffbe-6c9d-45bb-b0c1-7d7abf1051ae admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:34.681018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:34.681955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:34.682238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:34.682522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] PATH_INFO: `/v3/projects/11a218c85d0c4750963a4c6e1bde2259` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:34.684325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a218c85d0c4750963a4c6e1bde2259)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:34.686038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:34.867204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a218c85d0c4750963a4c6e1bde2259', 'request_id': 'req-ddd31e21-5c85-4d84-bf20-72802869ed0e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:34.878074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a218c85d0c4750963a4c6e1bde2259', 'request_id': 'req-ddd31e21-5c85-4d84-bf20-72802869ed0e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.000769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b5f71883e94de98f54c07a41f15497'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.001474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b5f71883e94de98f54c07a41f15497'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.016426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b5f71883e94de98f54c07a41f15497'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.062627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a218c85d0c4750963a4c6e1bde2259', 'request_id': 'req-ddd31e21-5c85-4d84-bf20-72802869ed0e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.078722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] The token cache is being invalidate because project 11a218c85d0c4750963a4c6e1bde2259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:35.079071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.079944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.081599 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 904/1808] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:34 2022] DELETE /identity/v3/projects/11a218c85d0c4750963a4c6e1bde2259 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:35.083136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.086167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.162385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd31e21-5c85-4d84-bf20-72802869ed0e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.167165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.167917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.168235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.168513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.169184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.170832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.182499 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 905/1809] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:35.183398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.185303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.199841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04194db4-917a-446b-8508-3bf6e7a7f294 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.204979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.205614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.205932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.206210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.206878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.208581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.228241 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 906/1810] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:35.228241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.228745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b5f71883e94de98f54c07a41f15497'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.231132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.238632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b5f71883e94de98f54c07a41f15497'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.247056 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 905/1811] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:34 2022] DELETE /identity/v3/users/07b5f71883e94de98f54c07a41f15497 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:35.247973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.249808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.250079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a7d-46dc-4c2b-a78f-ed85543ef4ee admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.339058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd343-747b-4565-a86b-c75cd691d213 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.342760 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.343511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.343881 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.344085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] PATH_INFO: `/v3/projects/6c2eb5277d964cd1a8383e3fb8565a62` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.345375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2eb5277d964cd1a8383e3fb8565a62)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.346035 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.346831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.347244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.347431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.347669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] PATH_INFO: `/v3/users/5a460601771d4ab58631f215300d0cb4` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.349185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a460601771d4ab58631f215300d0cb4)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.351504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.394365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:35.529278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2eb5277d964cd1a8383e3fb8565a62', 'request_id': 'req-767e921e-3790-4131-bb28-a6d582070cd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.535171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2eb5277d964cd1a8383e3fb8565a62', 'request_id': 'req-767e921e-3790-4131-bb28-a6d582070cd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.637701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2eb5277d964cd1a8383e3fb8565a62', 'request_id': 'req-767e921e-3790-4131-bb28-a6d582070cd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.655094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] The token cache is being invalidate because project 6c2eb5277d964cd1a8383e3fb8565a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:35.655408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.656180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.657818 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 907/1812] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:35 2022] DELETE /identity/v3/projects/6c2eb5277d964cd1a8383e3fb8565a62 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:35.661512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.668405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.707252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a460601771d4ab58631f215300d0cb4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.708311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a460601771d4ab58631f215300d0cb4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.732946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a460601771d4ab58631f215300d0cb4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.781537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e921e-3790-4131-bb28-a6d582070cd5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.789872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.790619 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.790992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.791400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.792188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.793905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.812979 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 908/1813] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:35.813159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a460601771d4ab58631f215300d0cb4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.813260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.822283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-357d2c7b-4dcc-4223-b33e-70f3198d60e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a460601771d4ab58631f215300d0cb4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:35.829475 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 906/1814] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:35 2022] DELETE /identity/v3/users/5a460601771d4ab58631f215300d0cb4 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:35.830174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.831185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.831493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.831801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.855488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c3145-1acf-4dc2-b083-b88e5e58f2b6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.871470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:35.874731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.942011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.942850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.942985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.943857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.944721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.946600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.960343 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 909/1815] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:35.961176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:35.963296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:35.981740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45da32-78bf-4b6a-aec1-2d6e445a25c2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:35.989655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:35.990761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:35.991253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:35.992660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] PATH_INFO: `/v3/users/0c9bac418f414127949146e54bdbfac0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:35.994584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9bac418f414127949146e54bdbfac0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:35.999679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:35.999949 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 907/1816] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:36.003759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.005949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.020759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d442a-9d87-4177-8f14-f7f1ae37953c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.027651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.028445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.028861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.029251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] PATH_INFO: `/v3/projects/20505445e12f43b79ebf73fdcf5c1a4f/users/70d58592082043cfb4af574fc9eed5c9/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.048898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:36.052181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20505445e12f43b79ebf73fdcf5c1a4f, user_id=70d58592082043cfb4af574fc9eed5c9, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.055047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:36.144696 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 908/1817] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:35 2022] PUT /identity/v3/projects/20505445e12f43b79ebf73fdcf5c1a4f/users/70d58592082043cfb4af574fc9eed5c9/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:36.145753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.253790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe30cd-5eeb-4862-a0ec-7516c29f8e8f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.259087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.259830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.260110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.260386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] PATH_INFO: `/v3/projects/8602887be2ac4a449cb2ae178ecc584c/users/baa13e4355db4e23ae552eb4203a2545/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.280909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8602887be2ac4a449cb2ae178ecc584c, user_id=baa13e4355db4e23ae552eb4203a2545, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.284575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:36.325630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9bac418f414127949146e54bdbfac0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:36.327011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9bac418f414127949146e54bdbfac0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:36.340644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9bac418f414127949146e54bdbfac0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:36.341693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9bac418f414127949146e54bdbfac0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:36.357979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-825b4526-0be9-4478-9d23-b286e5fc9fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9bac418f414127949146e54bdbfac0'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:36.456652 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 909/1818] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:36 2022] PUT /identity/v3/projects/8602887be2ac4a449cb2ae178ecc584c/users/baa13e4355db4e23ae552eb4203a2545/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:36.457731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.459415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.491145 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 910/1819] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:35 2022] DELETE /identity/v3/users/0c9bac418f414127949146e54bdbfac0 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:36.492857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a70917-7b4d-43dc-a544-bd430a4c1380 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.493252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a70917-7b4d-43dc-a544-bd430a4c1380 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.493771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a70917-7b4d-43dc-a544-bd430a4c1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.530406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-86a70917-7b4d-43dc-a544-bd430a4c1380 None None] MFA Rules not processed for user `70d58592082043cfb4af574fc9eed5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:36.531830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-86a70917-7b4d-43dc-a544-bd430a4c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.541112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b828c7-f46b-45b0-89d8-e6957c266fa2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.615172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.616197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.616810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.617259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.620567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.635701 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 911/1820] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:36.638359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.639031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.639279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.686317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] MFA Rules not processed for user `baa13e4355db4e23ae552eb4203a2545`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:36.688651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.722847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.724122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:36.743043 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 910/1821] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:36.746250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.749449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.799332 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 912/1822] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:36.800507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.802535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.820838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aae76-6a49-4890-b115-a397996890c3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.828256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.829023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.829430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.829846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] PATH_INFO: `/v3/projects/c8d7a72076044e808f68794474fb58a3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.843194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d7a72076044e808f68794474fb58a3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.848946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:36.906744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881482-5303-40f5-b083-8717198f4c37 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.917902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.919012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.919548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.920040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.937690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.940793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:36.958548 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 911/1823] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:36.960493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:36.962837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:36.981541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330e85-7eed-4d3b-b7b2-816dcabb5dd7 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:36.986683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:36.987544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:36.988021 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:36.988351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:36.989206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:36.992416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:37.043810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d7a72076044e808f68794474fb58a3', 'request_id': 'req-af14a539-3c92-4d4b-8636-9f86e7d07ec9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.086745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d7a72076044e808f68794474fb58a3', 'request_id': 'req-af14a539-3c92-4d4b-8636-9f86e7d07ec9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.102745 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 912/1824] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:36 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:37.104177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:37.171349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d7a72076044e808f68794474fb58a3', 'request_id': 'req-af14a539-3c92-4d4b-8636-9f86e7d07ec9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.212802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] The token cache is being invalidate because project c8d7a72076044e808f68794474fb58a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:37.213256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.215207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-af14a539-3c92-4d4b-8636-9f86e7d07ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.226147 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 913/1825] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:36 2022] DELETE /identity/v3/projects/c8d7a72076044e808f68794474fb58a3 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:37.235051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928c94-8688-42c0-805a-30327499e9d8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.235504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928c94-8688-42c0-805a-30327499e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.237068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2928c94-8688-42c0-805a-30327499e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.296388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c2928c94-8688-42c0-805a-30327499e9d8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:37.299188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c2928c94-8688-42c0-805a-30327499e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:37.339742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8bac0-133a-424e-993a-6d9c43144c86 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:37.348977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:37.349688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.350063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.350289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.351205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:37.355157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:37.409403 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 914/1826] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:37.418843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.419035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.419235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.443376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:37.446312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:37.462624 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 915/1827] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:37.466408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:37.468865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:37.497383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7291e8-cf03-4bf9-a184-9e0a3f0e71a9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:37.505891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:37.507225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.507827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.508075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] PATH_INFO: `/v3/projects/0fff6034ae6949139034d9d30e386166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.509577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fff6034ae6949139034d9d30e386166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:37.512317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:37.556224 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 913/1828] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:37 2022] POST /identity/v3/users => generated 289 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:37.556224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:37.574220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c30086-a36b-4795-99af-08b54d18ad3e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:37.584377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:37.586779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.587180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.587662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.588840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:37.589343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:37.592220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:37.619641 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 914/1829] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:37.619641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:37.620990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:37.643887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d295-ed2a-4b2f-a456-50a13bf3a040 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:37.651337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:37.652212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:37.652685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:37.653138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] PATH_INFO: `/v3/projects/91d84d0f5a664ad9bd037b414be0425a` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:37.655069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d84d0f5a664ad9bd037b414be0425a)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:37.658694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:37.758868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fff6034ae6949139034d9d30e386166', 'request_id': 'req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.766977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fff6034ae6949139034d9d30e386166', 'request_id': 'req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.862387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d84d0f5a664ad9bd037b414be0425a', 'request_id': 'req-930ded14-a4af-4be0-954e-6aab2d0a81fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.869893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d84d0f5a664ad9bd037b414be0425a', 'request_id': 'req-930ded14-a4af-4be0-954e-6aab2d0a81fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.877125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:37.878583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fff6034ae6949139034d9d30e386166', 'request_id': 'req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.881726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d84d0f5a664ad9bd037b414be0425a', 'request_id': 'req-930ded14-a4af-4be0-954e-6aab2d0a81fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.896688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] The token cache is being invalidate because project 0fff6034ae6949139034d9d30e386166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:37.896979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.897675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:37.899243 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 916/1830] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:37 2022] DELETE /identity/v3/projects/0fff6034ae6949139034d9d30e386166 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:37.900260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:37.901792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.003443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c88cc-cee6-4ffa-b6f2-6992268d6bac admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:38.005478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] The token cache is being invalidate because project 91d84d0f5a664ad9bd037b414be0425a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:38.005885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.006727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.008676 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 915/1831] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:37 2022] DELETE /identity/v3/projects/91d84d0f5a664ad9bd037b414be0425a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:38.009947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.011733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.104345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:38.105340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:38.105750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:38.106251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:38.107284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:38.110135 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:38.135877 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 917/1832] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:38.135877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.136670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ded14-a4af-4be0-954e-6aab2d0a81fc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:38.145558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:38.146392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:38.146903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:38.147520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:38.153242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:38.153242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:38.155355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.188519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c01bd-a6ba-48cc-b22f-e43271f1ee27 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:38.197197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:38.198244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:38.198721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:38.199204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] PATH_INFO: `/v3/users/db05ad6f184a40289f737ce797911ed8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:38.202819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db05ad6f184a40289f737ce797911ed8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:38.205587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:38.293933 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 916/1833] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:38 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:38.294865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.296704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.329649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:38.394681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fe824-5d75-4eac-ba71-f33c72d2f2d4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:38.402011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:38.402992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:38.403511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:38.403963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] PATH_INFO: `/v3/users/01c5db086b044ef2a8220913f7a248a4` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:38.405466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c5db086b044ef2a8220913f7a248a4)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:38.407719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:38.451798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db05ad6f184a40289f737ce797911ed8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.452291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db05ad6f184a40289f737ce797911ed8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.466011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db05ad6f184a40289f737ce797911ed8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.467128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db05ad6f184a40289f737ce797911ed8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.487947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db05ad6f184a40289f737ce797911ed8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.603249 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 918/1834] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:38 2022] DELETE /identity/v3/users/db05ad6f184a40289f737ce797911ed8 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:38.604849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.606549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.707750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e9cf-6db9-4d5f-97c0-6f27a7fec984 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:38.714073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:38.716141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:38.716681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:38.717063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] PATH_INFO: `/v3/projects/474759375d914e72a50276ce8013ebae/users/0a84434c2ff74e879886e906495fc779/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:38.738960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474759375d914e72a50276ce8013ebae, user_id=0a84434c2ff74e879886e906495fc779, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:38.744151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:38.745107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c5db086b044ef2a8220913f7a248a4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.745930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c5db086b044ef2a8220913f7a248a4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.766428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c5db086b044ef2a8220913f7a248a4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.890879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c5db086b044ef2a8220913f7a248a4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.896730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:38.905732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c5db086b044ef2a8220913f7a248a4'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:38.908995 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 919/1835] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:38 2022] PUT /identity/v3/projects/474759375d914e72a50276ce8013ebae/users/0a84434c2ff74e879886e906495fc779/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:38.911052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.914512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:38.926623 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 917/1836] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:38 2022] DELETE /identity/v3/users/01c5db086b044ef2a8220913f7a248a4 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:38.928796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:38.931162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:39.057863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ebe8-aee9-48da-a1ae-c696b3983277 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.085039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b7d0-5470-4f84-88d9-a26c6774752d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.091124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.091832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.092112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.092389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.093074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.095151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.108690 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 918/1837] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:39.113953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.132755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494facf-59fe-48d7-963f-3fe16bfb441f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.140064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.141452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.141888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.142333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.143137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.143253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.144514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.145144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.145725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] PATH_INFO: `/v3/projects/474759375d914e72a50276ce8013ebae/users/0a84434c2ff74e879886e906495fc779/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.146260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.149973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474759375d914e72a50276ce8013ebae, user_id=0a84434c2ff74e879886e906495fc779, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.155529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dbfcd-5996-4d6e-940b-7cbea9d23349 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.172250 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 919/1838] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:39 2022] PUT /identity/v3/projects/474759375d914e72a50276ce8013ebae/users/0a84434c2ff74e879886e906495fc779/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:39.186647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.187120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.187620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.221888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] MFA Rules not processed for user `0a84434c2ff74e879886e906495fc779`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:39.223214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:39.289104 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 920/1839] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:38 2022] POST /identity/v3/users => generated 292 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:39.294668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.310362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d988e7e-8df9-42a1-951e-41d32c2478e4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.370506 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 920/1840] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:39.388642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.392386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:39.428869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.429746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.430188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.431131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.432012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.434794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.451350 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 921/1841] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:39.458264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.476177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86afd-9506-41f8-8bd0-16746dce3de1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.483070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.484054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.484491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.485561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.488749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.489261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.510861 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 922/1842] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:39.510861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.510861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:39.531046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116f934-07ff-4e08-bf20-a66aca5350b5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.531613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:39.536064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.536739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.537026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.537305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] PATH_INFO: `/v3/projects/0a53708dc975481f88349132d5eeac4c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.539542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a53708dc975481f88349132d5eeac4c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.541512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.564516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e9e54-d405-40af-a833-ff331ed4f50d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.583364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.584944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.585511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.586136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.605563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.608068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.621975 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 921/1843] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:39.622815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.635732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2f1a8-fdcd-4220-90f0-f14a00ca20df service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.640918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:39.641796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:39.642083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:39.642573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] PATH_INFO: `/v3/projects/c1bf88a26c7a4c45995517a7a5dc87e8/users/e476c66d4e9240f0810b3ccd9fa04223/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:39.661062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bf88a26c7a4c45995517a7a5dc87e8, user_id=e476c66d4e9240f0810b3ccd9fa04223, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:39.664635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:39.820045 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 922/1844] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:39 2022] PUT /identity/v3/projects/c1bf88a26c7a4c45995517a7a5dc87e8/users/e476c66d4e9240f0810b3ccd9fa04223/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:39.822095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:39.828050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a53708dc975481f88349132d5eeac4c', 'request_id': 'req-d3505566-f708-4b9f-90b9-cac52e63b89e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:39.828503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:39.837636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a53708dc975481f88349132d5eeac4c', 'request_id': 'req-d3505566-f708-4b9f-90b9-cac52e63b89e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:39.957041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539dff-7f03-4f5a-a2e1-f72a439cd2cb admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:39.992515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a53708dc975481f88349132d5eeac4c', 'request_id': 'req-d3505566-f708-4b9f-90b9-cac52e63b89e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.013464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] The token cache is being invalidate because project 0a53708dc975481f88349132d5eeac4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:40.013766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.014637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d3505566-f708-4b9f-90b9-cac52e63b89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.016635 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 923/1845] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:39 2022] DELETE /identity/v3/projects/0a53708dc975481f88349132d5eeac4c => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:40.017551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d65c2-dead-4eb0-b091-68c0599108ce None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.017819 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d65c2-dead-4eb0-b091-68c0599108ce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.018103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d65c2-dead-4eb0-b091-68c0599108ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.052342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.053521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.053994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.054473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] PATH_INFO: `/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.060533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05fa29f0e2b46fd9d0279a9ddda372f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.062866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.065735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-711d65c2-dead-4eb0-b091-68c0599108ce None None] MFA Rules not processed for user `e476c66d4e9240f0810b3ccd9fa04223`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:40.067385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-711d65c2-dead-4eb0-b091-68c0599108ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.180776 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 924/1846] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:40.180776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.180776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.180776 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.215178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:40.218516 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.222994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05fa29f0e2b46fd9d0279a9ddda372f', 'request_id': 'req-c4bdb642-73ff-4a89-8349-87c76c8ad10b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.231064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05fa29f0e2b46fd9d0279a9ddda372f', 'request_id': 'req-c4bdb642-73ff-4a89-8349-87c76c8ad10b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.249757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05fa29f0e2b46fd9d0279a9ddda372f', 'request_id': 'req-c4bdb642-73ff-4a89-8349-87c76c8ad10b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.293438 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 925/1847] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:40.294581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.295934 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.311182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56522a0d-627f-496a-9540-249f0c27e3a0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.327850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.328551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.328882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.335703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.335703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.342644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.400629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] The token cache is being invalidate because project e05fa29f0e2b46fd9d0279a9ddda372f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:40.400900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.401641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c4bdb642-73ff-4a89-8349-87c76c8ad10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:40.404008 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 923/1848] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:39 2022] DELETE /identity/v3/projects/e05fa29f0e2b46fd9d0279a9ddda372f => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:40.405515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8539cc-4c43-42de-b4a0-2a3556a0e2f6 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.405811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8539cc-4c43-42de-b4a0-2a3556a0e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.406095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8539cc-4c43-42de-b4a0-2a3556a0e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.439479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6d8539cc-4c43-42de-b4a0-2a3556a0e2f6 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:40.441603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6d8539cc-4c43-42de-b4a0-2a3556a0e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.457278 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 924/1849] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:40.459676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.460172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.460706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.513683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:40.516138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.519494 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 926/1850] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:40.520960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.539882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:40.632423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3bf10-39b7-4572-b219-8b32ac798f87 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.641686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.642785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.643302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.643857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.645159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:40.651896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.651896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63e202-a849-490c-a208-7b57e378a5ef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.659785 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 925/1851] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:40.660782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.669408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.688970 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 927/1852] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:40.691186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.691729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.692241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.713572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6ebac-21d9-4d64-a779-f5a7cb48d641 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.724116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.725681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.726318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.737396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:40.737396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:40.737642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.737642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.741845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.743847 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 928/1853] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:40.744952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.765114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda3e3d-88d6-4f60-bb2d-a7981eea0ced None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.772269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.773249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.773718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.774304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.775190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.778052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.872480 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 926/1854] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:40.874234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.892939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0335-2710-415c-94d1-72a00814cb63 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.901179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.902257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.902721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.903220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.904521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:40.905070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.908425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.932122 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 927/1855] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:40.934511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.945762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce6147-c5c7-4213-b7f5-95e9d9992d7f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.950886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.951524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.951833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.952274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.952956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.954601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:40.965960 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 928/1856] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:40.968007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.979965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106f7e4-97af-49ea-a582-9da152312ff0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:40.987335 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 929/1857] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:40.987335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:40.990042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:40.991323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:40.991641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:40.992676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:40.993374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:40.995519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.072255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f60fc-507e-4ea5-ab39-ba754c8f1928 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.077431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.078121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.078402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.078800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] PATH_INFO: `/v3/projects/713d5467c7004df78dd82bbd90a36956/users/64264511818943cf8ff0196acc1da8a1/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.099121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713d5467c7004df78dd82bbd90a36956, user_id=64264511818943cf8ff0196acc1da8a1, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.102734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.116529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 929/1858] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:40 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:41.117529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.202188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bf162-20fa-449c-be2d-caff3dd8faf9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.202499 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 930/1859] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:40 2022] PUT /identity/v3/projects/713d5467c7004df78dd82bbd90a36956/users/64264511818943cf8ff0196acc1da8a1/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:41.203555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.288739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.289424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.289709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.289989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.290652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.293341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.316925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bf7b8-4f41-4a28-950e-1b5d7b081427 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.322092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.322723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.323238 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.323691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.324377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.326377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.439956 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 930/1860] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:41 2022] POST /identity/v3/users => generated 287 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:41.441084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.443028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:41.455632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d28c1-e66e-4548-859d-5ed0596b50e3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.461105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.461769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.462054 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.462331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] PATH_INFO: `/v3/users/baa13e4355db4e23ae552eb4203a2545` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.463578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa13e4355db4e23ae552eb4203a2545)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.465745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.485411 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 931/1861] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:41 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:41.486509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.497644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2fd9-e263-4c64-94fb-c87c41c357bf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.503412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.504084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.504366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.504645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.505317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.507027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.518561 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 932/1862] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:41.519505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.533859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b199-b6b1-4a45-8c89-014429853935 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.539840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.540704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.541091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.541453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.542222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.544101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.556539 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 933/1863] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:41.557424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.568244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc970-1797-417f-9720-21a5f264533f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.573801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.574462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.574739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.575011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.575697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.577325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.589329 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 934/1864] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:41.597208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.601896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9442563-3f0c-46ad-8959-48149001f21a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.607464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.608285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.608777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.609004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] PATH_INFO: `/v3/projects/713d5467c7004df78dd82bbd90a36956/users/64264511818943cf8ff0196acc1da8a1/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.611398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713d5467c7004df78dd82bbd90a36956, user_id=64264511818943cf8ff0196acc1da8a1, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.614104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.636488 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 935/1865] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:41 2022] PUT /identity/v3/projects/713d5467c7004df78dd82bbd90a36956/users/64264511818943cf8ff0196acc1da8a1/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:41.638250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.713479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:41.715715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa13e4355db4e23ae552eb4203a2545'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:41.718572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa13e4355db4e23ae552eb4203a2545'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:41.752647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa13e4355db4e23ae552eb4203a2545'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:41.764162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c98627-773d-4a2d-a54e-2b8c3c827721 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.771236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.772480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.772962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.774226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.774644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.776769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.801074 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 936/1866] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:41.801074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.809379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:41.837267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f6692-431d-46da-beb2-4be26edf2243 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:41.845193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:41.846453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.846453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.846775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.847483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:41.849610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ff1aa-33dd-419e-b56a-c4f89e803bfd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:41.862144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa13e4355db4e23ae552eb4203a2545'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:41.870163 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 937/1867] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:41.871936 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa13e4355db4e23ae552eb4203a2545'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:41.873140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:41.873307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:41.873697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:41.880340 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 931/1868] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:41 2022] DELETE /identity/v3/users/baa13e4355db4e23ae552eb4203a2545 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:41.881979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:41.898851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:41.920137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] MFA Rules not processed for user `64264511818943cf8ff0196acc1da8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:41.922199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:41.996002 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400391f-d6e9-497b-8df7-bfda18437bdc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.003367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.004487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.005463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.005952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] PATH_INFO: `/v3/projects/b3ac92b862ae4d988409b01c3adfef75/users/19714f6672d345a38370effe826a3d2a/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.024138 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 938/1869] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:42.028545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ac92b862ae4d988409b01c3adfef75, user_id=19714f6672d345a38370effe826a3d2a, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.029087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.032991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ff997-d6aa-4094-ba9b-58ab12cdcdf6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.041059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ab2a0-fa8a-48fe-8a44-be55505334d7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.046368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.048588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.048588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.048588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] PATH_INFO: `/v3/projects/354eead4e8f34baf96b0048e4ae0758f/users/5c97a67748254c1c800f34ea1c63f30f/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.066040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354eead4e8f34baf96b0048e4ae0758f, user_id=5c97a67748254c1c800f34ea1c63f30f, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.069428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.168846 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 932/1870] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:41 2022] PUT /identity/v3/projects/b3ac92b862ae4d988409b01c3adfef75/users/19714f6672d345a38370effe826a3d2a/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:42.170631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b6db4-dc3f-4837-ada1-71e53318623f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.170988 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b6db4-dc3f-4837-ada1-71e53318623f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.171331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b6db4-dc3f-4837-ada1-71e53318623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.202813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2f8b6db4-dc3f-4837-ada1-71e53318623f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:42.205050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2f8b6db4-dc3f-4837-ada1-71e53318623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.208655 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 939/1871] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:42 2022] PUT /identity/v3/projects/354eead4e8f34baf96b0048e4ae0758f/users/5c97a67748254c1c800f34ea1c63f30f/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:42.210007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.212925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.286383 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 933/1872] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:42.288911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b5bb-01bd-4aab-ad79-57b76619bb92 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.289331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b5bb-01bd-4aab-ad79-57b76619bb92 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.289643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b5bb-01bd-4aab-ad79-57b76619bb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.312167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8ef4b5bb-01bd-4aab-ad79-57b76619bb92 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:42.313607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8ef4b5bb-01bd-4aab-ad79-57b76619bb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.321401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43b19f-77c7-43ad-805e-98a696ebd7c8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.323416 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 934/1873] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:42.324739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.325209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.325490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.332073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.333509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.333999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.334584 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.337747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.367740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] MFA Rules not processed for user `5c97a67748254c1c800f34ea1c63f30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:42.369027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.429338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.431385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.443814 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 940/1874] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:42.444707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.446118 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.465040 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 935/1875] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:42.465040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.465040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:42.539137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f605-d79b-4409-a533-5713f7fce9b7 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.546099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.547141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.547664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.548280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.569146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.572265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.587830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462d90c-0719-44c2-a811-63fd14d742c4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.589434 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 941/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:42.590711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.600878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.600878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.600878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.600878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.605418 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278b081-45a7-4162-ad4d-edd5457c66e9 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.613912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.615044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.616183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.617056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.617791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.619002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.621637 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.623160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.645235 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 936/1877] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:42.661721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:42.772651 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 942/1878] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:42 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:42.775131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.793660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867c7f-d026-49f8-b68c-63ffbb798c30 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:42.870692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:42.871536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:42.872194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:42.872194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:42.873294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:42.875636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:42.978367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:42.986761 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 937/1879] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:42 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:42.994148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8542d41-6e4c-4e79-889d-e851a37908b0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.001280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.006696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.006696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.006696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.006696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.006696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.019385 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 943/1880] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:43.024201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.035572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a8eb-3388-400a-971f-685505543fce admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.041883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.042854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.043362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.043762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] PATH_INFO: `/v3/projects/af171354a78a4d6f80cc303fe33edd5d/users/68809f944cf44649b61c9bd91ef20289/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.064112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af171354a78a4d6f80cc303fe33edd5d, user_id=68809f944cf44649b61c9bd91ef20289, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.066938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.160291 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 938/1881] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:43 2022] PUT /identity/v3/projects/af171354a78a4d6f80cc303fe33edd5d/users/68809f944cf44649b61c9bd91ef20289/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:43.165811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.177607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9568-cb6c-4cb2-a5c9-dcbe887190f4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.270552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.275811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.275811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.275811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.276126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.279256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d1c8c-d912-4bff-8b87-19216bc011d7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.292268 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 944/1882] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:43.297118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.319293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d425-dfbf-472e-b5b7-656350839a09 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.327233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.328431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.328941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.329444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] PATH_INFO: `/v3/projects/af171354a78a4d6f80cc303fe33edd5d/users/68809f944cf44649b61c9bd91ef20289/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.333065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af171354a78a4d6f80cc303fe33edd5d, user_id=68809f944cf44649b61c9bd91ef20289, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.338827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.354673 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 939/1883] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:43 2022] PUT /identity/v3/projects/af171354a78a4d6f80cc303fe33edd5d/users/68809f944cf44649b61c9bd91ef20289/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:43.367536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.367900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.368184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.392130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.396213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:43.516583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] MFA Rules not processed for user `68809f944cf44649b61c9bd91ef20289`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:43.522165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:43.569887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bd5f2-f8cb-4c12-8934-1bc14fff56c4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.578033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.579369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.580084 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.581466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] PATH_INFO: `/v3/projects/8602887be2ac4a449cb2ae178ecc584c` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.583765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8602887be2ac4a449cb2ae178ecc584c)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.588485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.607669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:43.684858 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 945/1884] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:43.705388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.727947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:43.785647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7cf2-2e00-4ac8-8505-6fc46bf04c0f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:43.799088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8602887be2ac4a449cb2ae178ecc584c', 'request_id': 'req-b677f721-2e38-4011-ae16-302db50062e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:43.811479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8602887be2ac4a449cb2ae178ecc584c', 'request_id': 'req-b677f721-2e38-4011-ae16-302db50062e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:43.825623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8602887be2ac4a449cb2ae178ecc584c', 'request_id': 'req-b677f721-2e38-4011-ae16-302db50062e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:43.862637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:43.863343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:43.864305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:43.864586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:43.909005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] The token cache is being invalidate because project 8602887be2ac4a449cb2ae178ecc584c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:43.909682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:43.910931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:43.913640 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 940/1885] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:43 2022] DELETE /identity/v3/projects/8602887be2ac4a449cb2ae178ecc584c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:43.920260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:43.922430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:43.950117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:43.952201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:43.966831 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 946/1886] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:44.018479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677f721-2e38-4011-ae16-302db50062e6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:44.031540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:44.032979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:44.033898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:44.034591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] PATH_INFO: `/v3/users/0a84434c2ff74e879886e906495fc779` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:44.036384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a84434c2ff74e879886e906495fc779)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:44.038823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:44.092108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:44.111139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c0c0f-9eb2-477f-bd30-b2ba21e112b3 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:44.117790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:44.118741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:44.119172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:44.119526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] PATH_INFO: `/v3/users/5c97a67748254c1c800f34ea1c63f30f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:44.121173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c97a67748254c1c800f34ea1c63f30f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:44.124479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:44.247482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a84434c2ff74e879886e906495fc779'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.249320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a84434c2ff74e879886e906495fc779'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.258234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:44.279393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a84434c2ff74e879886e906495fc779'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.310614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:44.414148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c97a67748254c1c800f34ea1c63f30f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.415165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c97a67748254c1c800f34ea1c63f30f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.417891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a84434c2ff74e879886e906495fc779'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.429376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c97a67748254c1c800f34ea1c63f30f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.434096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c97a67748254c1c800f34ea1c63f30f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.439275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2d38c023-a313-4a5c-bad9-675dbaca5cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a84434c2ff74e879886e906495fc779'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.454430 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 941/1887] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:43 2022] DELETE /identity/v3/users/0a84434c2ff74e879886e906495fc779 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:44.461634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c97a67748254c1c800f34ea1c63f30f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:44.621775 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 947/1888] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:44 2022] DELETE /identity/v3/users/5c97a67748254c1c800f34ea1c63f30f => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:44.630760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:44.632932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:44.633778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:44.723269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:44.725668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:44.852947 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 942/1889] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:44.868596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:44.871295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:44.969523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883ffc0-e9db-48cf-adbb-0084d96666c6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:44.976538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:44.977597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:44.978092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:44.978578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.000047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:45.003033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1632e-ab5f-491b-b2d6-f10febded7a3 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:45.025015 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 948/1890] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:45.139421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:45.141365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.168500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430890-5c56-401e-bce1-5171a4487462 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:45.173889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:45.174564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.174848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.175125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] PATH_INFO: `/v3/projects/474759375d914e72a50276ce8013ebae` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.176420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474759375d914e72a50276ce8013ebae)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:45.179015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:45.218066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.218515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.218980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.245580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:45.263228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] MFA Rules not processed for user `19714f6672d345a38370effe826a3d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:45.265490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.388882 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 949/1891] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:45.407227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:45.409211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.429463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474759375d914e72a50276ce8013ebae', 'request_id': 'req-c8b4450d-1e8b-4112-ac40-5c8e56520c94'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:45.438096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474759375d914e72a50276ce8013ebae', 'request_id': 'req-c8b4450d-1e8b-4112-ac40-5c8e56520c94'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:45.446328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474759375d914e72a50276ce8013ebae', 'request_id': 'req-c8b4450d-1e8b-4112-ac40-5c8e56520c94'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:45.507841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f598e-928c-4d8d-8638-fa0d99b762b4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:45.541358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] The token cache is being invalidate because project 474759375d914e72a50276ce8013ebae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:45.542181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:45.543819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c8b4450d-1e8b-4112-ac40-5c8e56520c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:45.551703 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 943/1892] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:45 2022] DELETE /identity/v3/projects/474759375d914e72a50276ce8013ebae => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:45.569944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8afd6-b9a7-4f2a-89d6-6011c60dfff0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.572185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8afd6-b9a7-4f2a-89d6-6011c60dfff0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.575857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8afd6-b9a7-4f2a-89d6-6011c60dfff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.619970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:45.620957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.621412 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.621860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.628081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.644679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-add8afd6-b9a7-4f2a-89d6-6011c60dfff0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:45.644679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-add8afd6-b9a7-4f2a-89d6-6011c60dfff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.775847 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 944/1893] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:45.784986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.785931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.789862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.790186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:45.791694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:45.821191 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 950/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:45.843197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:45.846613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:45.874037 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 945/1895] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:45.875299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:45.902583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb81a0-0429-4dab-b53e-82f71c59bb77 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:45.918153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:45.919313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.920997 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.921541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.923349 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:45.924509 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:45.928941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:45.955985 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 951/1896] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:45.962960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:45.980917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379da980-324b-4fa8-976f-2f9cbb125f8f None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:45.985622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:45.992700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:45.993906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:45.994534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:45.995153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:45.996457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.000965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.152378 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 946/1897] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:45 2022] POST /identity/v3/projects => generated 391 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:46.154515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.169999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75249f4d-1f21-4106-a9a8-180c3b4b5e22 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.183933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.186496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.267863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.268502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.268786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.269066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.269739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.271864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.309065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7efc7-08e9-4edc-a6d0-32594d1f3172 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.318678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.319755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.320218 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.320795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] PATH_INFO: `/v3/projects/354eead4e8f34baf96b0048e4ae0758f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.323055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354eead4e8f34baf96b0048e4ae0758f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.326615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.372599 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 952/1898] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:46 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:46.378953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.392022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af1126-6235-47d2-9beb-d0dbfa76df6c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.397530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.398197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.398475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.398783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.399458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.401164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.425109 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 953/1899] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:46.430131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.441011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b403ae-434a-4cc1-8514-790acffd43ab admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.446413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.447057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.447390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.447839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.448404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.454038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.467846 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 954/1900] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:46.469172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.476635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:46.480943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c783937-071b-4a30-afd3-68d16b6e3000 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.482521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354eead4e8f34baf96b0048e4ae0758f', 'request_id': 'req-2e022a5e-6516-433d-8a33-0e43ef295c75'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:46.490186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354eead4e8f34baf96b0048e4ae0758f', 'request_id': 'req-2e022a5e-6516-433d-8a33-0e43ef295c75'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:46.503171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354eead4e8f34baf96b0048e4ae0758f', 'request_id': 'req-2e022a5e-6516-433d-8a33-0e43ef295c75'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:46.553515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.554199 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.554486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.554774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] PATH_INFO: `/v3/projects/5570d9a488724f3592fb70163fd543a5/users/7fa23bca3b30440db5df90da639e0dd3/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.571894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5570d9a488724f3592fb70163fd543a5, user_id=7fa23bca3b30440db5df90da639e0dd3, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.574622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256cb73-629d-45b1-95df-8c0c509f591f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.737360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] The token cache is being invalidate because project 354eead4e8f34baf96b0048e4ae0758f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:46.737360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:46.738172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:46.739876 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 947/1901] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:46 2022] DELETE /identity/v3/projects/354eead4e8f34baf96b0048e4ae0758f => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:46.742086 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 955/1902] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:46 2022] PUT /identity/v3/projects/5570d9a488724f3592fb70163fd543a5/users/7fa23bca3b30440db5df90da639e0dd3/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:46.744014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.745642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.759947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022a5e-6516-433d-8a33-0e43ef295c75 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.769020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.769020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.769020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.799873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] MFA Rules not processed for user `7fa23bca3b30440db5df90da639e0dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:46.801544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.836405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.837070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.837357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.837727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] PATH_INFO: `/v3/users/68809f944cf44649b61c9bd91ef20289` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.839045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68809f944cf44649b61c9bd91ef20289)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.841143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.895299 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 956/1903] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:46.909165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.911966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.937994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e559d-c4af-46bc-bae0-55b180988e30 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.947779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.949069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.949583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.950057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.951230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.955223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.092806 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 957/1904] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:46 2022] POST /identity/v3/projects => generated 391 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:47.093784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.098512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68809f944cf44649b61c9bd91ef20289'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:47.101914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68809f944cf44649b61c9bd91ef20289'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:47.134081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68809f944cf44649b61c9bd91ef20289'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:47.194763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33fb0-f0f8-4f6c-a411-32d6351c9138 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:47.202004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:47.202742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.203026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.203434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.204683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:47.208492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.228219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68809f944cf44649b61c9bd91ef20289'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:47.238632 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68809f944cf44649b61c9bd91ef20289'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:47.246568 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 948/1905] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:46 2022] DELETE /identity/v3/users/68809f944cf44649b61c9bd91ef20289 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:47.322209 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 958/1906] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:47 2022] POST /identity/v3/users => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:47.326856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.329407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:47.413914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd898d5-2214-447c-840c-e19d813ce805 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:47.418910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:47.419696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.419917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.420196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.420866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:47.422526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.435034 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 949/1907] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:47.439920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.452696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877a3a3-658d-4d35-b396-ba4749d08d68 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:47.458378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:47.459134 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.459528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.459888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] PATH_INFO: `/v3/projects/84c6c34d91fe4372a1c7a311171b900d/users/d1cbd138813640109ec7c372da4dee19/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.477221 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c6c34d91fe4372a1c7a311171b900d, user_id=d1cbd138813640109ec7c372da4dee19, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:47.479838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.595275 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 959/1908] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:47 2022] PUT /identity/v3/projects/84c6c34d91fe4372a1c7a311171b900d/users/d1cbd138813640109ec7c372da4dee19/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:47.601776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.620859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8902-69f7-41fd-9a0f-5f0a2c3f5a1b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:47.710622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:47.711256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.711656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.711886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.712601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:47.714221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f8cd9-438b-45ba-a5bd-11391563200f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.726678 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 950/1909] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:47.731267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.743117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb79db-ccf2-4e4c-ab41-b75c9f178fa3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:47.749284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:47.749976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.750261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.750537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] PATH_INFO: `/v3/projects/84c6c34d91fe4372a1c7a311171b900d/users/d1cbd138813640109ec7c372da4dee19/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.752780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c6c34d91fe4372a1c7a311171b900d, user_id=d1cbd138813640109ec7c372da4dee19, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:47.755545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:47.767257 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 960/1910] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:47 2022] PUT /identity/v3/projects/84c6c34d91fe4372a1c7a311171b900d/users/d1cbd138813640109ec7c372da4dee19/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:47.780044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:47.780384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:47.780615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:47.812736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] MFA Rules not processed for user `d1cbd138813640109ec7c372da4dee19`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:47.813979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:47.833081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.843404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:47.920469 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 951/1911] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:47 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:47.970841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:47.973407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:47.997300 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f80d4-9ad4-49fe-849d-d44a9b5cb930 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:48.004368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:48.005511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.006077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.006802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] PATH_INFO: `/v3/projects/af171354a78a4d6f80cc303fe33edd5d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.008910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af171354a78a4d6f80cc303fe33edd5d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:48.012700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:48.101536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a2425-c457-4889-8996-fdb82d23a59e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:48.106714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:48.107413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.107793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.108116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.126282 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:48.129228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30858149-378a-4975-80d9-a6f1d83b70d2 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:48.152021 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 952/1912] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:48.188219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af171354a78a4d6f80cc303fe33edd5d', 'request_id': 'req-e1fe4187-2c15-4a9e-9a89-f190841899c7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:48.207323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af171354a78a4d6f80cc303fe33edd5d', 'request_id': 'req-e1fe4187-2c15-4a9e-9a89-f190841899c7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:48.258312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:48.317338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24af4-e2e8-439e-8820-6062ae1b352d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.317795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24af4-e2e8-439e-8820-6062ae1b352d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.318762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24af4-e2e8-439e-8820-6062ae1b352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.346838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-60c24af4-e2e8-439e-8820-6062ae1b352d None None] MFA Rules not processed for user `19714f6672d345a38370effe826a3d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:48.348410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-60c24af4-e2e8-439e-8820-6062ae1b352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:48.350553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af171354a78a4d6f80cc303fe33edd5d', 'request_id': 'req-e1fe4187-2c15-4a9e-9a89-f190841899c7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:48.395442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] The token cache is being invalidate because project af171354a78a4d6f80cc303fe33edd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:48.398685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:48.402901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:48.419333 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 961/1913] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:47 2022] DELETE /identity/v3/projects/af171354a78a4d6f80cc303fe33edd5d => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:48.422322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:48.426958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:48.483576 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 953/1914] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:48.490083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0b5a-0598-470b-83c2-e44e1cf517ed None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.490584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0b5a-0598-470b-83c2-e44e1cf517ed None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.491037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0b5a-0598-470b-83c2-e44e1cf517ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.531025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0d4b0b5a-0598-470b-83c2-e44e1cf517ed None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:48.533611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0d4b0b5a-0598-470b-83c2-e44e1cf517ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:48.552886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe4187-2c15-4a9e-9a89-f190841899c7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:48.560326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:48.562253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.562868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.563473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.571917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:48.583051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:48.707499 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 954/1915] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:48.712694 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.713463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.714112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.722762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:48.725569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:48.753205 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 962/1916] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:48.755365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:48.758486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:48.775048 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 955/1917] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:48.781451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:48.800470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e76f54-a2bd-4d34-b640-6d904af468b3 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:48.809320 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:48.810363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.810841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.811304 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.812802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:48.813318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:48.816409 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:48.845749 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 963/1918] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:48.847547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:48.894889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eff73-29ca-498c-bd1d-16227ba8962e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:48.916202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:48.917757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:48.919057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:48.919979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:48.923526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:48.929409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:49.053390 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 956/1919] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:48 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:49.058020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:49.069395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84046119-7466-44ed-97a3-e16d964bae5f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:49.134293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:49.140674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.141162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.141371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.142399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:49.145993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:49.344396 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 964/1920] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:49 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:49.348486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:49.360462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef92f24-40ec-4a93-a8a4-abee590e37d5 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:49.365967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:49.366654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.366932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.367202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.367889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:49.369530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:49.380970 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 957/1921] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:49.385940 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:49.399162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195ad9-3404-4178-bba2-4986402a08da admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:49.405578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:49.406289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.406695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.407099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.407882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:49.409625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e07bb-897b-4ebc-b24a-d7b071fa3387 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:49.424233 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 965/1922] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:49.429625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:49.441535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27bad6-38a5-4834-aac5-4e7945871e7c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:49.447806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:49.448585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.449032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.449395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] PATH_INFO: `/v3/projects/f86e7881f15446a4be9c24448902b4bc/users/89157374212146af9b61e12e91277e62/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.469185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86e7881f15446a4be9c24448902b4bc, user_id=89157374212146af9b61e12e91277e62, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:49.472142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:49.571403 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 958/1923] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:49 2022] PUT /identity/v3/projects/f86e7881f15446a4be9c24448902b4bc/users/89157374212146af9b61e12e91277e62/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:49.582295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.583378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.584506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.642882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] MFA Rules not processed for user `89157374212146af9b61e12e91277e62`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:49.642882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:49.746808 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 966/1924] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:49.751945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:49.767861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:49.854012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ec3e-e857-47ab-a62a-89e93f7c810a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:49.858176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:49.859450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:49.860088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:49.860546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:49.861530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:49.865383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:50.170170 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 959/1925] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:49 2022] POST /identity/v3/projects => generated 381 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:10:50.172069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:50.187222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a67da-bb3f-4bf9-9502-00966a9ed027 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:50.363751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:50.366560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:50.367857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:50.368890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:50.370508 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:50.382771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:50.417203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:50.436463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:50.528562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a792e-1b62-4622-a9f4-301a9e179cad admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:50.533922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:50.534635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:50.534912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:50.535184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:50.537261 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:50.636366 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 967/1926] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:10:50 2022] POST /identity/v3/users => generated 297 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:50.637750 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:50.639421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:50.640459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:50.662925 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 960/1927] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:50.681691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4b3e4-e2d5-4c3a-bf0d-d2713eaf0cae admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:50.695625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:50.697152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:50.698109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:50.698723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:50.699889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:50.703346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:50.718444 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 968/1928] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:10:50.726648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:50.751527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78e6e3-6e36-4142-8dce-5827c26fa4d2 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:50.774789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:50.785793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:50.785793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:50.785793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] PATH_INFO: `/v3/projects/8e0b3aed411f4e27ab98e87038b8b757/users/66c1060dcf0747d88bfeacc85be2a703/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:50.798677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0b3aed411f4e27ab98e87038b8b757, user_id=66c1060dcf0747d88bfeacc85be2a703, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:50.801585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:50.923971 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 961/1929] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:50 2022] PUT /identity/v3/projects/8e0b3aed411f4e27ab98e87038b8b757/users/66c1060dcf0747d88bfeacc85be2a703/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:50.929166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:50.942611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5db9-a11a-4ca5-83f9-ee049007c0e6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:50.960362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:51.028915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:51.063611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:51.065116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:51.066282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:51.066832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:51.068323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:51.071559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702239f-84fa-44d0-b360-257b58f173a5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:51.093596 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 969/1930] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:10:51.098026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:51.113464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c2e5-fab8-47f2-877d-5904641480d5 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:51.120527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:51.120825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:51.121425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:51.121934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] PATH_INFO: `/v3/projects/8e0b3aed411f4e27ab98e87038b8b757/users/66c1060dcf0747d88bfeacc85be2a703/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:51.124780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0b3aed411f4e27ab98e87038b8b757, user_id=66c1060dcf0747d88bfeacc85be2a703, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:51.128776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:51.145793 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 962/1931] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:10:51 2022] PUT /identity/v3/projects/8e0b3aed411f4e27ab98e87038b8b757/users/66c1060dcf0747d88bfeacc85be2a703/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:51.160964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:51.161687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:51.162970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:51.211690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] MFA Rules not processed for user `66c1060dcf0747d88bfeacc85be2a703`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:51.213794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:51.336876 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 970/1932] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:51.354671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:51.357833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:51.443070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060daf-388d-457b-8035-684c26272354 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:51.448058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:51.448822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:51.449114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:51.449388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:51.461410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:51.463199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:51.477529 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 963/1933] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:51.960763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:52.192178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:52.926378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:53.377903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:53.527098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:54.331420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:54.336883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b938f5-6e9f-4ead-abb6-24dd96324fe9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:54.346697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66c1060dcf0747d88bfeacc85be2a703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0b3aed411f4e27ab98e87038b8b757', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:54.347363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:54.347558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:54.350750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:54.358013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:54.358013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:54.358013 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:54.377765 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 971/1934] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:10:54.386295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:54.397476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6f7d-9cae-49c2-8f42-b67ae429b171 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:54.404292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66c1060dcf0747d88bfeacc85be2a703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0b3aed411f4e27ab98e87038b8b757', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:54.405150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:54.408408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:54.408408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:54.408408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:54.409970 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:54.581289 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 964/1935] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:10:54 2022] POST /identity/v3/projects => generated 353 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:10:54.582665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:54.601803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a9d51-8aee-4e99-bdf0-09d48c0b9b81 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:54.730219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66c1060dcf0747d88bfeacc85be2a703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0b3aed411f4e27ab98e87038b8b757', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:54.731143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:54.731612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:54.732058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:54.732907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:10:54.733293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:54.735325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:54.752774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:54.767705 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 972/1936] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:10:54.924307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:55.355709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:55.371219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1e7a5-75e9-4a4f-adb4-4a4a16913a00 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:55.377004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66c1060dcf0747d88bfeacc85be2a703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0b3aed411f4e27ab98e87038b8b757', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:55.377762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:55.378105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:55.378386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] PATH_INFO: `/v3/projects/81a8f6470b3447cea2becc5127f9aad9` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:55.380602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorizing `identity:delete_project(project_id=81a8f6470b3447cea2becc5127f9aad9)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:55.382856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:55.648409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a8f6470b3447cea2becc5127f9aad9', 'request_id': 'req-f40f9c3c-831c-4b44-88c4-0c799053ad86'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:55.651218 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a8f6470b3447cea2becc5127f9aad9', 'request_id': 'req-f40f9c3c-831c-4b44-88c4-0c799053ad86'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:55.665965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a8f6470b3447cea2becc5127f9aad9', 'request_id': 'req-f40f9c3c-831c-4b44-88c4-0c799053ad86'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:55.769085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] The token cache is being invalidate because project 81a8f6470b3447cea2becc5127f9aad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:55.774421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:55.774421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:55.774421 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 965/1937] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:55 2022] DELETE /identity/v3/projects/81a8f6470b3447cea2becc5127f9aad9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:56.053988 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:56.599072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:57.358596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:57.364347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:57.397640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:57.400051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:57.561920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f9c3c-831c-4b44-88c4-0c799053ad86 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:57.569769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:57.570734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:57.571176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:57.571914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] PATH_INFO: `/v3/users/89157374212146af9b61e12e91277e62` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:57.573771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89157374212146af9b61e12e91277e62)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:57.576832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:57.577332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaceb3f-1b5f-4391-a24f-c20ba6609d27 tempest-VolumeQuotaClassesTest-1632783074 tempest-VolumeQuotaClassesTest-1632783074-project-admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:57.586427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:57.587727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:57.588355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:57.588920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] PATH_INFO: `/v3/users/7fa23bca3b30440db5df90da639e0dd3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:57.592403 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa23bca3b30440db5df90da639e0dd3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:57.597604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:57.875167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa23bca3b30440db5df90da639e0dd3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.875379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89157374212146af9b61e12e91277e62'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.876132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89157374212146af9b61e12e91277e62'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.876326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa23bca3b30440db5df90da639e0dd3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.890385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa23bca3b30440db5df90da639e0dd3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.891537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa23bca3b30440db5df90da639e0dd3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.892039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89157374212146af9b61e12e91277e62'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:57.910971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-075ab339-a728-4131-8a82-3e9007fee7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa23bca3b30440db5df90da639e0dd3'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:58.004684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89157374212146af9b61e12e91277e62'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:58.014452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a95e206d-4546-4d9e-ae2f-10fac60a9034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89157374212146af9b61e12e91277e62'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:58.022958 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 966/1938] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:57 2022] DELETE /identity/v3/users/89157374212146af9b61e12e91277e62 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:58.028194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:58.030539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:58.038617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:58.038617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:58.058366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:58.059759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.149372 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 973/1939] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:57 2022] DELETE /identity/v3/users/7fa23bca3b30440db5df90da639e0dd3 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:58.155836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:58.156308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:58.156721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:58.170856 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 967/1940] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:58.184531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:58.186627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.189967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:58.191858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.270905 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 974/1941] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:10:58.282329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:58.287971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e0d18-a216-42aa-ad86-eca8d537922c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:58.381298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bdcd3-61ca-473c-8db3-4dfbfe44190c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:58.385433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:58.386089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:58.386359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:58.386835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:58.387945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:58.389141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:58.389640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:58.390148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:58.399458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:58.401480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:58.408220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:58.412467 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:58.415455 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 975/1942] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:58.431635 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 968/1943] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:58.885456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:59.278444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:59.281132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:59.297812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9a40b-b5d3-494e-944f-a211297beb49 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:59.308998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:59.308998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:59.308998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:59.308998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] PATH_INFO: `/v3/users/70d58592082043cfb4af574fc9eed5c9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:59.309779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d58592082043cfb4af574fc9eed5c9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:59.310064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:59.483901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:59.485793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:59.504995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3149c-d748-4731-9e44-d78f867fbb87 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:59.512256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:59.513365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:59.513932 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:59.514763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] PATH_INFO: `/v3/projects/f86e7881f15446a4be9c24448902b4bc` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:59.516924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86e7881f15446a4be9c24448902b4bc)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:59.521572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:59.571928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d58592082043cfb4af574fc9eed5c9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.572875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d58592082043cfb4af574fc9eed5c9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.581650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d58592082043cfb4af574fc9eed5c9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.582417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d58592082043cfb4af574fc9eed5c9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.606754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d58592082043cfb4af574fc9eed5c9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.699954 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 976/1944] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:59 2022] DELETE /identity/v3/users/70d58592082043cfb4af574fc9eed5c9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:10:59.702609 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:59.704383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:59.726483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86e7881f15446a4be9c24448902b4bc', 'request_id': 'req-5a47611a-f197-4a6b-b5b4-912011fb54e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.734701 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86e7881f15446a4be9c24448902b4bc', 'request_id': 'req-5a47611a-f197-4a6b-b5b4-912011fb54e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.744507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86e7881f15446a4be9c24448902b4bc', 'request_id': 'req-5a47611a-f197-4a6b-b5b4-912011fb54e6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.748890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:59.795354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785c126-da60-4378-847f-4ef23a949621 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:59.839115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] The token cache is being invalidate because project f86e7881f15446a4be9c24448902b4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:59.839423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.844158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5a47611a-f197-4a6b-b5b4-912011fb54e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:59.846932 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 969/1945] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:59 2022] DELETE /identity/v3/projects/f86e7881f15446a4be9c24448902b4bc => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:59.848928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:59.851991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:59.852462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:59.882789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:59.884389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:59.890154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:59.891833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:59.891833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:59.891833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] PATH_INFO: `/v3/projects/5570d9a488724f3592fb70163fd543a5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:59.893799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5570d9a488724f3592fb70163fd543a5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:59.896736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:59.974598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:59.985593 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 970/1946] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:59.987206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:59.990171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:00.014258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913262b4-78bc-4883-b1a0-88dc43448f51 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:00.023275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:00.024427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:00.024949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:00.025451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] PATH_INFO: `/v3/users/66c1060dcf0747d88bfeacc85be2a703` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:00.027732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c1060dcf0747d88bfeacc85be2a703)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:00.032030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:00.093335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5570d9a488724f3592fb70163fd543a5', 'request_id': 'req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.105826 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5570d9a488724f3592fb70163fd543a5', 'request_id': 'req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.191901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5570d9a488724f3592fb70163fd543a5', 'request_id': 'req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.213967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] The token cache is being invalidate because project 5570d9a488724f3592fb70163fd543a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:00.214518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.215419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.217471 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 977/1947] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:59 2022] DELETE /identity/v3/projects/5570d9a488724f3592fb70163fd543a5 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:00.218621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:00.220464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:00.241319 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c1060dcf0747d88bfeacc85be2a703'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.242387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c1060dcf0747d88bfeacc85be2a703'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.267718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c1060dcf0747d88bfeacc85be2a703'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.340152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e903aa8-f3b4-4252-9f59-6c33a396e4ea admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:00.344441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:00.352486 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:00.352862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:00.353064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:00.355758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:00.416746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c1060dcf0747d88bfeacc85be2a703'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.437220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c1060dcf0747d88bfeacc85be2a703'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.451695 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 971/1948] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:59 2022] DELETE /identity/v3/users/66c1060dcf0747d88bfeacc85be2a703 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:00.453542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:00.455625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:00.486804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:00.489657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:00.515387 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 978/1949] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:00.516766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:00.518633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:00.534543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6320e-d3dc-4bac-bcc1-caadcdebd5e7 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:00.609271 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659271a-010b-4036-807b-737f7be00919 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:00.618117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:00.619211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:00.619812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:00.620310 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] PATH_INFO: `/v3/users/d1cbd138813640109ec7c372da4dee19` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:00.623154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cbd138813640109ec7c372da4dee19)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:00.627172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:00.640029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:00.641518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:00.642042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:00.642469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] PATH_INFO: `/v3/users/e476c66d4e9240f0810b3ccd9fa04223` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:00.643930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e476c66d4e9240f0810b3ccd9fa04223)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:00.646512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:00.976159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e476c66d4e9240f0810b3ccd9fa04223'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.978162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e476c66d4e9240f0810b3ccd9fa04223'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.983368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cbd138813640109ec7c372da4dee19'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.984645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cbd138813640109ec7c372da4dee19'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:00.998500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e476c66d4e9240f0810b3ccd9fa04223'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.000138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e476c66d4e9240f0810b3ccd9fa04223'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.041503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ddd3679c-8c01-4455-b8ef-087f470bf8e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e476c66d4e9240f0810b3ccd9fa04223'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.045157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cbd138813640109ec7c372da4dee19'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.223020 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 979/1950] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:00 2022] DELETE /identity/v3/users/e476c66d4e9240f0810b3ccd9fa04223 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:01.230617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:01.231241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:01.231737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:01.263903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:01.265901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.309619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cbd138813640109ec7c372da4dee19'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.329391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cbd138813640109ec7c372da4dee19'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.342706 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 972/1951] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:00 2022] DELETE /identity/v3/users/d1cbd138813640109ec7c372da4dee19 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:01.384947 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 980/1952] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:01.400087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:01.403265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.537884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb96964-cf2c-4bf6-805d-8ac9afbab71a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:01.540021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:01.542054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.543800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:01.544560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:01.544851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:01.545181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:01.567542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966fb53-c8cb-4ce7-a730-920cd0f9886d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:01.647974 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:01.689439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:01.693165 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:01.711077 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 973/1953] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:01.712072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:01.713677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.735966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5924-91f0-4350-a1bd-2558fb3a500f service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:01.742561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:01.743256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:01.743672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:01.743976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] PATH_INFO: `/v3/projects/8e0b3aed411f4e27ab98e87038b8b757` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:01.745889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0b3aed411f4e27ab98e87038b8b757)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:01.748518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:01.751913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:01.753056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:01.754425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:01.754880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] PATH_INFO: `/v3/projects/20505445e12f43b79ebf73fdcf5c1a4f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:01.758399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=20505445e12f43b79ebf73fdcf5c1a4f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:01.763019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:01.954358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0b3aed411f4e27ab98e87038b8b757', 'request_id': 'req-afdb19e8-2931-4ad8-8909-44a452a7fd76'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.961178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20505445e12f43b79ebf73fdcf5c1a4f', 'request_id': 'req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.965088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:01.965861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0b3aed411f4e27ab98e87038b8b757', 'request_id': 'req-afdb19e8-2931-4ad8-8909-44a452a7fd76'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.972478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20505445e12f43b79ebf73fdcf5c1a4f', 'request_id': 'req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:01.979048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0b3aed411f4e27ab98e87038b8b757', 'request_id': 'req-afdb19e8-2931-4ad8-8909-44a452a7fd76'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.096277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20505445e12f43b79ebf73fdcf5c1a4f', 'request_id': 'req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.113869 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] The token cache is being invalidate because project 20505445e12f43b79ebf73fdcf5c1a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:02.114303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.115354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b65cbbeb-0a4a-4f3e-84cf-839706bc4eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.117348 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 981/1954] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:01 2022] DELETE /identity/v3/projects/20505445e12f43b79ebf73fdcf5c1a4f => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:02.123162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e50fe-f0db-4fcb-b7fb-e3e86c1a2727 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.123465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e50fe-f0db-4fcb-b7fb-e3e86c1a2727 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.123847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e50fe-f0db-4fcb-b7fb-e3e86c1a2727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.166719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-168e50fe-f0db-4fcb-b7fb-e3e86c1a2727 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:02.166719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-168e50fe-f0db-4fcb-b7fb-e3e86c1a2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.196660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] The token cache is being invalidate because project 8e0b3aed411f4e27ab98e87038b8b757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:02.198315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.200212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-afdb19e8-2931-4ad8-8909-44a452a7fd76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.205471 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 974/1955] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:01 2022] DELETE /identity/v3/projects/8e0b3aed411f4e27ab98e87038b8b757 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:02.214120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.216811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.228737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.274753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:02.278538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.311460 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 982/1956] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:02.323176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.323912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.324393 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.362941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:02.367756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.382488 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 983/1957] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:02.383526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.397169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a821663-a7f1-44e2-9a81-c9ae6b3250eb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:02.412205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:02.413778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.414257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.414710 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.415705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:02.416132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:02.419026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9079a71-eb07-42db-8e06-43fedacc4402 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:02.422450 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 975/1958] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:02.425783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.429823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.447901 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 984/1959] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:02.447901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.447901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.447901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.456841 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79edc1d-95a7-46ab-ad0a-a2814a8cfb16 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:02.465567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:02.466564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.467051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.467546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] PATH_INFO: `/v3/projects/84c6c34d91fe4372a1c7a311171b900d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.469244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:02.469661 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c6c34d91fe4372a1c7a311171b900d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:02.470679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.473535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:02.480194 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 985/1960] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:02.480772 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.493381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e0d2-c5ee-4af5-ad1b-2102508a73c1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:02.498813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:02.499660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.499946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.500226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.500891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:02.502839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:02.733611 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 986/1961] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/projects => generated 363 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:02.735245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.744563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c6c34d91fe4372a1c7a311171b900d', 'request_id': 'req-5294c0c9-ee3f-4e56-a494-38fca7cd223c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.758127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c6c34d91fe4372a1c7a311171b900d', 'request_id': 'req-5294c0c9-ee3f-4e56-a494-38fca7cd223c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.772820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c6c34d91fe4372a1c7a311171b900d', 'request_id': 'req-5294c0c9-ee3f-4e56-a494-38fca7cd223c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.850982 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1271e-8606-48d4-a652-af0b29727d24 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:02.856414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:02.857227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.857517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.857807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.858478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:02.858795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:02.860669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:02.877898 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 987/1962] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:02.881244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.887941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.918168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] The token cache is being invalidate because project 84c6c34d91fe4372a1c7a311171b900d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:02.918675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.919378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:02.921012 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 976/1963] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:02 2022] DELETE /identity/v3/projects/84c6c34d91fe4372a1c7a311171b900d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:02.923322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:02.929425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:02.949350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0cbf5-2b33-4590-b4ff-528f19a6b90d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:02.963576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:02.964565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:02.965003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:02.965430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] PATH_INFO: `/v3/projects/c1bf88a26c7a4c45995517a7a5dc87e8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:02.967991 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1bf88a26c7a4c45995517a7a5dc87e8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:02.970648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.044487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c0c9-ee3f-4e56-a494-38fca7cd223c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.054461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.055573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.056098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.072132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.072132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.072132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.210739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1bf88a26c7a4c45995517a7a5dc87e8', 'request_id': 'req-7dfc02ae-ad5d-4860-97a5-4c8621380527'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.217811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1bf88a26c7a4c45995517a7a5dc87e8', 'request_id': 'req-7dfc02ae-ad5d-4860-97a5-4c8621380527'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.263614 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 977/1964] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:02 2022] POST /identity/v3/users => generated 282 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:03.264981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.267334 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.285895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3438f12-043c-4a4e-b4bd-5c41fdfb941f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.324956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1bf88a26c7a4c45995517a7a5dc87e8', 'request_id': 'req-7dfc02ae-ad5d-4860-97a5-4c8621380527'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.341301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] The token cache is being invalidate because project c1bf88a26c7a4c45995517a7a5dc87e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:03.341587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.342581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7dfc02ae-ad5d-4860-97a5-4c8621380527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.348476 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 988/1965] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:02 2022] DELETE /identity/v3/projects/c1bf88a26c7a4c45995517a7a5dc87e8 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:03.348673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.349870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.350272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.366837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.367865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.368333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.368795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.369922 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.373830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e4f3b-e8f3-480d-9588-e251a5b15f96 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.385410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:03.387454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.400734 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 989/1966] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:03.401687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.403195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.417460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9a4e-86e7-4eca-b8c4-052407aec226 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.422532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.423174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.423481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.423827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.424510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.426148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.442326 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 990/1967] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:03.443719 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.445813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.464338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68119f8-fbed-460e-bbb7-6e8f8c67fecc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.469851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.470583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.470867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.471153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] PATH_INFO: `/v3/users/64264511818943cf8ff0196acc1da8a1` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.472429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64264511818943cf8ff0196acc1da8a1)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.475141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.532982 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 978/1968] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:03 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:03.535791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.537233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.537389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.546584 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:03.573189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:03.574444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.645551 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 979/1969] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:03.646833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.666018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6c84d-a1f2-40fd-bc81-7f86d63348ad None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.675804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.676293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.676537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.677016 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.678865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.682151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.710263 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 980/1970] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:03.711243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.715399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.743853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64264511818943cf8ff0196acc1da8a1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.744775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64264511818943cf8ff0196acc1da8a1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.746846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb20-c084-4785-8df2-a6619abdb1f0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.753995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.754885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.755097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.755418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.756263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.758688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.759807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64264511818943cf8ff0196acc1da8a1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.761773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64264511818943cf8ff0196acc1da8a1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.785243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64264511818943cf8ff0196acc1da8a1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:03.869862 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 981/1971] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:03 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:03.871498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.873709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.886594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6643-23d0-423b-9cdf-4940a40b1674 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:03.892018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:03.892662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:03.892979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:03.893257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:03.893947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:03.894275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:03.896474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:03.920600 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 982/1972] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:03.920600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.920967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.942856 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 991/1973] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:03 2022] DELETE /identity/v3/users/64264511818943cf8ff0196acc1da8a1 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:03.944186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:03.946603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:03.959424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dee12-44c2-4462-9f69-ffb53dbddbfe admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.107817 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8920ff-8406-4e42-952c-4682157597a1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.118901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.119685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.120243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.120749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.122613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.126709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.135449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.136989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.137651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.138219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] PATH_INFO: `/v3/projects/7b63baebedae48bc97e663daf54fb710/users/fad56d680ba2475caf240311bf625e99/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.151154 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 992/1974] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:04.152858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.179439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9899-9ab4-49ab-afd1-34b66afd18d3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.188179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.189215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.189716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.190214 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.191759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.195195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4355fc7-867d-4383-88b6-95422193b7aa admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.196915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b63baebedae48bc97e663daf54fb710, user_id=fad56d680ba2475caf240311bf625e99, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.200996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.343567 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 983/1975] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:03 2022] PUT /identity/v3/projects/7b63baebedae48bc97e663daf54fb710/users/fad56d680ba2475caf240311bf625e99/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:04.344886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.378102 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 993/1976] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:04 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:04.380673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.381152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.381816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.451973 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] MFA Rules not processed for user `fad56d680ba2475caf240311bf625e99`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:04.454855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:04.471178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073155-cf44-45fc-94bc-03dc25f74c17 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.590679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.591488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.591814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.592120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.592842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.594603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.608398 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 984/1977] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:04.612962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.616262 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 994/1978] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:04.618094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.633487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb7f9-e4a7-4c0a-94a5-4f8cfac0ecb0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.641818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.643153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.643153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.643153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.643826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.643937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3cf17-1eb1-4260-821e-9d842da943b8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.645856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.656616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.657700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.658058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.658416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] PATH_INFO: `/v3/projects/35f0ae8f61364a59bcf8c24b80232406/users/915c12c39d4d4c49b981def7ec88d52a/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.691822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f0ae8f61364a59bcf8c24b80232406, user_id=915c12c39d4d4c49b981def7ec88d52a, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.695797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.801423 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 985/1979] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:04 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:04.804427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.807679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:04.841344 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 995/1980] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:04 2022] PUT /identity/v3/projects/35f0ae8f61364a59bcf8c24b80232406/users/915c12c39d4d4c49b981def7ec88d52a/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:04.842538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:04.844556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:04.907783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e851c-6b2c-49d8-982b-c870fb88dd6f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.935671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584c1ce-4ef0-4245-90f0-efe7e03c2a56 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:04.943313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.944365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.944821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.945274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] PATH_INFO: `/v3/projects/713d5467c7004df78dd82bbd90a36956` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.947296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=713d5467c7004df78dd82bbd90a36956)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:04.950861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:04.978339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:04.979100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:04.979390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:04.979711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:04.981246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:05.054587 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.056191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.067401 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 986/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:05.068504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.080647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0603d8-26c4-4bba-b9bc-e4ffee73a464 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.089456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713d5467c7004df78dd82bbd90a36956', 'request_id': 'req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:05.092942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:05.101649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713d5467c7004df78dd82bbd90a36956', 'request_id': 'req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:05.158713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.159598 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.160211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.160552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.162202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.164931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610dca6-5846-4ab9-8a8e-be5ce4a5b4bd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.181825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713d5467c7004df78dd82bbd90a36956', 'request_id': 'req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:05.183664 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 987/1982] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:05.184009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13276-3f50-4ecf-b89f-7134ff3c84c5 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.184354 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13276-3f50-4ecf-b89f-7134ff3c84c5 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.184756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13276-3f50-4ecf-b89f-7134ff3c84c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.205526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] The token cache is being invalidate because project 713d5467c7004df78dd82bbd90a36956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:05.206219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:05.207211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:05.215006 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 996/1983] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:04 2022] DELETE /identity/v3/projects/713d5467c7004df78dd82bbd90a36956 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:05.216293 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.219278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.223489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-61a13276-3f50-4ecf-b89f-7134ff3c84c5 None None] MFA Rules not processed for user `915c12c39d4d4c49b981def7ec88d52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:05.227029 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-61a13276-3f50-4ecf-b89f-7134ff3c84c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.239143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35070-fdd0-4e3b-9d36-68f3a43c2cb8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.245464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.246155 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.246503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.246789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.247771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.249794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccfd13-b058-46d8-b96b-2398d4c4a18d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.266473 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 997/1984] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:05.268366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.280065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.284359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.333748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:05.335058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.346757 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 998/1985] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:05.359534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.362153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3609d7-55a2-4892-b905-132c9fb4c46f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.364204 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 988/1986] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:05.366954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.367328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.367692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.372024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.373149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.373551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.373730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] PATH_INFO: `/v3/projects/29ac87c5d8a8498da52eec8cc6d3ae7c/users/adef118180214eb5a5304511c03f546d/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.394132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ac87c5d8a8498da52eec8cc6d3ae7c, user_id=adef118180214eb5a5304511c03f546d, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.398009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.404983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:05.406276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.419312 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 989/1987] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:05.420173 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.421848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.435872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460a23c-51b2-43ac-b170-41f597023093 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.443400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.443819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.444102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.444380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.445042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.447020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c15b4-9ad0-4685-8b0c-2688dbfe4daf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.560397 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 999/1988] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:05 2022] PUT /identity/v3/projects/29ac87c5d8a8498da52eec8cc6d3ae7c/users/adef118180214eb5a5304511c03f546d/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:05.561922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.627570 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 990/1989] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:05.629137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.629406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.629682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.684300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] MFA Rules not processed for user `adef118180214eb5a5304511c03f546d`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:05.685673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:05.714162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf1f0-c5f9-4b81-b564-731d53d23a45 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.808262 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 991/1990] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:05.809335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.870202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.871017 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.871503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.871970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.873391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:05.873927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.876295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.901977 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1000/1991] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:05.903073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:05.933518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951183-c32e-4212-a746-78dbc398ae19 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.939316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461653f-8c50-49ca-aca7-799fd3ed1a6e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:05.940212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.941053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.941514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.942054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.943535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.947129 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:05.953003 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:05.954125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:05.954673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:05.955478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:05.957787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:05.962837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.108624 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 992/1992] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/users => generated 298 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:06.109844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.118253 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1001/1993] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:05 2022] POST /identity/v3/projects => generated 389 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:06.119777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.167349 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb87b-d447-4322-a0e4-579a62460909 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.401146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e2d9-814b-4a73-b46c-554fcb62d821 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.416481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.419722 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.420046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.420469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.421277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.423338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.433951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.435014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.435484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.435994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.437187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.438489 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1002/1994] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:06.440378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.442615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.453519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce61d39-2678-4745-939f-66d81a9aa215 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.459771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.460805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.460805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.461160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.461702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.463773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97302265-3dcd-4047-a5d0-09c30c6f49af admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.613057 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1003/1995] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:06 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:06.613782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.615215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.615408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.629525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 993/1996] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:06 2022] POST /identity/v3/projects => generated 377 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:06.635172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.657381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] MFA Rules not processed for user `a4a4230574c846a9b4f7edd240317ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:06.659626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:06.789569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83832fc7-05ec-4571-bfa6-1a6aab66272a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.801514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.803844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.804454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.804999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] PATH_INFO: `/v3/projects/ed2d8394d6c84974aa1d459f8a758668/users/543a1bb7a31c4937b3c9f1b821f34f52/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.807086 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1004/1997] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:06.807841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.821971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789129-501a-4aa3-8df9-6a24747bb2c0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.829142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.829980 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.830351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.830669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.831354 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.833834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.834973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2d8394d6c84974aa1d459f8a758668, user_id=543a1bb7a31c4937b3c9f1b821f34f52, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.840651 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.847947 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1005/1998] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:06.849538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:06.861310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3633-f706-4f45-b35e-3fa5ef07fd7a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:06.868677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:06.869479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:06.870468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:06.870468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:06.870873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:06.873614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:06.978388 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 994/1999] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:06 2022] PUT /identity/v3/projects/ed2d8394d6c84974aa1d459f8a758668/users/543a1bb7a31c4937b3c9f1b821f34f52/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:06.980409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.033368 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1006/2000] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:06 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:07.033368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.043645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5e991-54e0-430c-bdb9-01d0f3c6140e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.108279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991297-0b23-4cef-84eb-50234effb034 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.114605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.115953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.115953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.116252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] PATH_INFO: `/v3/projects/1a9869d1142349caaead92ab15463d67/users/dbdb4128ec504f54a7361f6ff923b820/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.135937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9869d1142349caaead92ab15463d67, user_id=dbdb4128ec504f54a7361f6ff923b820, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.139387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.149848 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.152176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.152680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.153171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.155225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.157677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.178950 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1007/2001] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:07.181047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.200840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cc16-c726-4ded-a35b-661ecdc67e23 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.207254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.207947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.208868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.209156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.210031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.213039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.234518 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1008/2002] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:07.236420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.255906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b2f7-ef2f-4f66-8217-2c77fbdd775c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.284028 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 995/2003] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:06 2022] PUT /identity/v3/projects/1a9869d1142349caaead92ab15463d67/users/dbdb4128ec504f54a7361f6ff923b820/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:07.285107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.387389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:07.394622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2d44b-f639-46b6-a4ea-d03850bee60c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.398662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.399514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.399831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.400108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] PATH_INFO: `/v3/projects/ed2d8394d6c84974aa1d459f8a758668/users/543a1bb7a31c4937b3c9f1b821f34f52/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.400759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.401450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.401743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.402113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.402406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2d8394d6c84974aa1d459f8a758668, user_id=543a1bb7a31c4937b3c9f1b821f34f52, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.402991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.405830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.406449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.424116 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 996/2004] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:07.424116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.424495 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1009/2005] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:07 2022] PUT /identity/v3/projects/ed2d8394d6c84974aa1d459f8a758668/users/543a1bb7a31c4937b3c9f1b821f34f52/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:07.430411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.439818 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b836-d68e-400d-affa-ff255adc41d6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.523445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:07.577098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.577878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.578384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.579200 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.580039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7b420-f67e-4ec4-8a68-c57e891a7405 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.580213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.582832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a0b5e-addb-47b4-ac15-f65dbaa1b246 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.590290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.591294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.591639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.591926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] PATH_INFO: `/v3/projects/61f624796bac4252ad0e0b5d117284f1/users/dba0acf6b5cd4069bb149b9eca4aedb7/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.601459 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 997/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:07.607636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7d34-7a37-4773-a74a-762adc15c979 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.608082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7d34-7a37-4773-a74a-762adc15c979 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.608601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7d34-7a37-4773-a74a-762adc15c979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.617088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f624796bac4252ad0e0b5d117284f1, user_id=dba0acf6b5cd4069bb149b9eca4aedb7, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.622547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.648574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d17a7d34-7a37-4773-a74a-762adc15c979 None None] MFA Rules not processed for user `543a1bb7a31c4937b3c9f1b821f34f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:07.649854 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d17a7d34-7a37-4773-a74a-762adc15c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:07.715647 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1010/2007] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:07 2022] PUT /identity/v3/projects/61f624796bac4252ad0e0b5d117284f1/users/dba0acf6b5cd4069bb149b9eca4aedb7/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:07.715647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.782904 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 998/2008] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:07.784606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57fe1-4ec1-4e94-b932-4db652426ed2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.784919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57fe1-4ec1-4e94-b932-4db652426ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.785196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57fe1-4ec1-4e94-b932-4db652426ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.833987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c4d57fe1-4ec1-4e94-b932-4db652426ed2 None None] MFA Rules not processed for user `dba0acf6b5cd4069bb149b9eca4aedb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:07.836565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c4d57fe1-4ec1-4e94-b932-4db652426ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:07.840490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01ade-c44b-4d46-86c2-217442a37454 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:07.849618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:07.850825 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.851258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.851726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] PATH_INFO: `/v3/projects/1a9869d1142349caaead92ab15463d67/users/dbdb4128ec504f54a7361f6ff923b820/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.855907 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9869d1142349caaead92ab15463d67, user_id=dbdb4128ec504f54a7361f6ff923b820, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:07.861533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:07.883423 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1011/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:07 2022] PUT /identity/v3/projects/1a9869d1142349caaead92ab15463d67/users/dbdb4128ec504f54a7361f6ff923b820/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:07.892797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:07.892797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:07.941069 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 999/2010] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:07.943291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:07.943745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:07.944085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:07.980246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] MFA Rules not processed for user `dbdb4128ec504f54a7361f6ff923b820`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:07.981504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:08.087519 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1000/2011] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:08.088779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:08.100672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33592de7-1660-4193-9a5e-3f954cfcf8fe admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:08.106510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:08.107473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:08.108138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:08.108325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:08.205188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:08.206879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:08.222527 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1012/2012] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:08.259994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d34150-4273-45e7-81ca-a00b06e42e42 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:08.266411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:08.268051 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:08.268537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:08.268814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:08.269502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:08.271785 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:08.292750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:08.303799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1800a-4884-4c64-b165-0d82c68d10cc service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:08.308877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:08.309545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:08.309827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:08.310530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:08.325724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:08.329128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:08.349225 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1013/2013] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:08.826690 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1001/2014] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:08 2022] POST /identity/v3/projects => generated 377 bytes in 735 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:08.828241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:08.840182 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d982-47d1-42ed-b525-d23de3eeca88 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:08.905262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:08.905977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:08.906204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:08.906481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:08.907160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:08.909219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.178944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.219949 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1014/2015] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:08 2022] POST /identity/v3/users => generated 289 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:09.226922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.237306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e696d3a-dda8-4398-bef8-7f27f2784be2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.242867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.244033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.244033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.244033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.244792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.246429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.258927 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1015/2016] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:09.259221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff02d-b68b-4170-b13f-16adf37e0ba2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.263080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.264199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.265000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.265293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.265580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.279572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71188216-3c4f-406b-8a7d-d5dc6359baec admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.285309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.286192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.286865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.287058 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.287784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.289488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.302357 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1016/2017] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:09.306211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.316924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c97875-39db-4987-8a0e-318780a563db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.323711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.324572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.324861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.325144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] PATH_INFO: `/v3/projects/2c4d498132c04cec9b72f18a6e23ef57/users/b3125a76e7824f2b8c7cbe81c7837495/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.348038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4d498132c04cec9b72f18a6e23ef57, user_id=b3125a76e7824f2b8c7cbe81c7837495, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.350838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.351777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.354417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66498df-dd4f-43b8-a79d-66a24c350907 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.369066 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1002/2018] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:09.482089 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1017/2019] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:09 2022] PUT /identity/v3/projects/2c4d498132c04cec9b72f18a6e23ef57/users/b3125a76e7824f2b8c7cbe81c7837495/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:09.494102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.494102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.499364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.527328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] MFA Rules not processed for user `b3125a76e7824f2b8c7cbe81c7837495`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:09.528656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:09.540073 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.668066 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1003/2020] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:09.668566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:09.689497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a001d15-419f-4d16-ad93-269ac7f4330a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.696239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf56d28-fa0c-4285-8bd0-3f94a210ace4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:09.699630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.700777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.702226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.703431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:09.704267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] PATH_INFO: `/v3/users/adef118180214eb5a5304511c03f546d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.706985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adef118180214eb5a5304511c03f546d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.707522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:09.708965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:09.709450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:09.710336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:09.718305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.721502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:09.893166 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1004/2021] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:09 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:09.898275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.024668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac34498-ac6a-4963-b1e6-ea988486b22a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.029963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.030655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.030942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.031253 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.031959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.033962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.104236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adef118180214eb5a5304511c03f546d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.105213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adef118180214eb5a5304511c03f546d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.117190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adef118180214eb5a5304511c03f546d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.118371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adef118180214eb5a5304511c03f546d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.141274 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1005/2022] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:09 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:10.146859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.149549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:10.150753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2838437f-40df-4db8-9447-27a775ef9695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adef118180214eb5a5304511c03f546d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.173611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30576bf2-d9da-4640-a90c-4cf05c60b5b4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.181879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.182895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.183527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.184033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.185142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.188073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.209959 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1006/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:10.214589 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.225928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e3be2-8d16-4f80-8c49-a05048b3631b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.231033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.231763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.234835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.234835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] PATH_INFO: `/v3/projects/e552720985df4365b6e97afc139ed397/users/77238492505b434b9fb73491aae13b72/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.249980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e552720985df4365b6e97afc139ed397, user_id=77238492505b434b9fb73491aae13b72, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.252808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.266013 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1018/2024] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:09 2022] DELETE /identity/v3/users/adef118180214eb5a5304511c03f546d => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:10.272007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.272155 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.272364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.295151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:10.297795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:10.373852 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1019/2025] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:10.374804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.381292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:10.399142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec94bb4-d08d-4787-95a0-db9974368288 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.413527 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1007/2026] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:10 2022] PUT /identity/v3/projects/e552720985df4365b6e97afc139ed397/users/77238492505b434b9fb73491aae13b72/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:10.414829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.418476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:10.520010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10d358-5281-4554-9b96-7d683aaabc63 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.520010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.520010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.520010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.520643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.535952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.537121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.537904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.538600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] PATH_INFO: `/v3/users/915c12c39d4d4c49b981def7ec88d52a` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.540545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=915c12c39d4d4c49b981def7ec88d52a)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.544572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.620732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.623867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.641162 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1008/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:10.642061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.652851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcf220-b81a-4c93-9139-a760c34ee8be service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.658653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.659439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.659884 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.660490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.661375 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.664941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.682439 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1009/2028] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:10.686864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.700338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef060e6-eefa-41be-b010-9d1a9e82fe61 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:10.706091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:10.706787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.707258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.707548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] PATH_INFO: `/v3/projects/e552720985df4365b6e97afc139ed397/users/77238492505b434b9fb73491aae13b72/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.711289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e552720985df4365b6e97afc139ed397, user_id=77238492505b434b9fb73491aae13b72, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:10.718507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0029e-adf9-44aa-88ac-fce62b4b0dfc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:10.734213 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1010/2029] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:10 2022] PUT /identity/v3/projects/e552720985df4365b6e97afc139ed397/users/77238492505b434b9fb73491aae13b72/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:10.748924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:10.749556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:10.749887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:10.783670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] MFA Rules not processed for user `77238492505b434b9fb73491aae13b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:10.785004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:10.849580 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:10.875228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '915c12c39d4d4c49b981def7ec88d52a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.877444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '915c12c39d4d4c49b981def7ec88d52a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.892470 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1011/2030] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:10.899747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '915c12c39d4d4c49b981def7ec88d52a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.899747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '915c12c39d4d4c49b981def7ec88d52a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.915686 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e77e009-0bf6-4055-b317-48f887c3d78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '915c12c39d4d4c49b981def7ec88d52a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:10.924698 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:10.932194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:11.045429 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1020/2031] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:10 2022] DELETE /identity/v3/users/915c12c39d4d4c49b981def7ec88d52a => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:11.049436 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:11.049758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:11.050007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:11.066983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb99c6-c310-4b7d-b9e8-ab4bbb8f1392 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:11.100812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:11.102787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:11.124947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:11.173178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:11.173938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:11.174232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:11.174534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:11.179059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:11.202331 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1021/2032] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:11.213653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:11.215125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:11.334547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ecd38-fd92-4c08-bb3d-f61418943ffa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:11.353033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:11.353692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:11.353946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:11.355993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:11.357104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:11.357616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:11.364954 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1012/2033] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:11.365905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:11.367402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:11.387229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:11.388803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:11.402552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d920-4be1-4544-9e3c-8d3152279d77 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:11.412948 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1022/2034] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:11.413298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:11.414857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:11.415380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:11.415957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] PATH_INFO: `/v3/projects/29ac87c5d8a8498da52eec8cc6d3ae7c` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:11.418312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ac87c5d8a8498da52eec8cc6d3ae7c)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:11.423955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:11.655977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ac87c5d8a8498da52eec8cc6d3ae7c', 'request_id': 'req-46efc3f6-ecd6-4f86-b577-3dff49f04963'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:11.673109 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ac87c5d8a8498da52eec8cc6d3ae7c', 'request_id': 'req-46efc3f6-ecd6-4f86-b577-3dff49f04963'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:11.686706 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ac87c5d8a8498da52eec8cc6d3ae7c', 'request_id': 'req-46efc3f6-ecd6-4f86-b577-3dff49f04963'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:11.835362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] The token cache is being invalidate because project 29ac87c5d8a8498da52eec8cc6d3ae7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:11.836101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:11.836923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:11.839111 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1013/2035] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:11 2022] DELETE /identity/v3/projects/29ac87c5d8a8498da52eec8cc6d3ae7c => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:11.844390 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:11.858047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81dfd-add5-4163-bf9d-6796a34c970a service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:11.945912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:11.946707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:11.947120 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:11.947830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] PATH_INFO: `/v3/users/dbdb4128ec504f54a7361f6ff923b820` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:11.950104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdb4128ec504f54a7361f6ff923b820)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:11.952938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:12.049624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:12.057197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:12.087186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efc3f6-ecd6-4f86-b577-3dff49f04963 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:12.095037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:12.095939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:12.096555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:12.097153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] PATH_INFO: `/v3/projects/35f0ae8f61364a59bcf8c24b80232406` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:12.099286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f0ae8f61364a59bcf8c24b80232406)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:12.101967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:12.188895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdb4128ec504f54a7361f6ff923b820'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.193428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdb4128ec504f54a7361f6ff923b820'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.208750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdb4128ec504f54a7361f6ff923b820'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.210589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdb4128ec504f54a7361f6ff923b820'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.249487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdb4128ec504f54a7361f6ff923b820'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.256830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f0ae8f61364a59bcf8c24b80232406', 'request_id': 'req-45334c58-7325-4c5f-8213-f667c7393dc3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.270727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f0ae8f61364a59bcf8c24b80232406', 'request_id': 'req-45334c58-7325-4c5f-8213-f667c7393dc3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.279380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f0ae8f61364a59bcf8c24b80232406', 'request_id': 'req-45334c58-7325-4c5f-8213-f667c7393dc3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.402673 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1023/2036] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:11 2022] DELETE /identity/v3/users/dbdb4128ec504f54a7361f6ff923b820 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:12.618879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] The token cache is being invalidate because project 35f0ae8f61364a59bcf8c24b80232406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:12.621410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.623643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:12.627398 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1014/2037] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:12 2022] DELETE /identity/v3/projects/35f0ae8f61364a59bcf8c24b80232406 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:12.634083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:12.640004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:12.792316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09962d12-ff61-4672-a38c-f3b29a0f86f3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:12.798439 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:12.799158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:12.799448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:12.799862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] PATH_INFO: `/v3/users/543a1bb7a31c4937b3c9f1b821f34f52` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:12.801164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=543a1bb7a31c4937b3c9f1b821f34f52)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:12.803965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:12.852326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:13.021165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '543a1bb7a31c4937b3c9f1b821f34f52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.021837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '543a1bb7a31c4937b3c9f1b821f34f52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.030667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '543a1bb7a31c4937b3c9f1b821f34f52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.033820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '543a1bb7a31c4937b3c9f1b821f34f52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.043665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:13.045978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:13.054244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9b92b293-de3f-4003-aafd-f11115fb0cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '543a1bb7a31c4937b3c9f1b821f34f52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.142656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45334c58-7325-4c5f-8213-f667c7393dc3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:13.148361 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:13.149164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:13.149503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:13.149825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] PATH_INFO: `/v3/projects/1a9869d1142349caaead92ab15463d67` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:13.151266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9869d1142349caaead92ab15463d67)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:13.153411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:13.178403 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1024/2038] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:12 2022] DELETE /identity/v3/users/543a1bb7a31c4937b3c9f1b821f34f52 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:13.373320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9869d1142349caaead92ab15463d67', 'request_id': 'req-b9e4b004-5903-4dfd-8c20-a44a7547d226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.385008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9869d1142349caaead92ab15463d67', 'request_id': 'req-b9e4b004-5903-4dfd-8c20-a44a7547d226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.397782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9869d1142349caaead92ab15463d67', 'request_id': 'req-b9e4b004-5903-4dfd-8c20-a44a7547d226'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.543507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:13.585234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] The token cache is being invalidate because project 1a9869d1142349caaead92ab15463d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:13.585999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.587741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b9e4b004-5903-4dfd-8c20-a44a7547d226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:13.590054 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1015/2039] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:13 2022] DELETE /identity/v3/projects/1a9869d1142349caaead92ab15463d67 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:13.598264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd554-bb94-486d-955d-348383178022 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:13.599208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd554-bb94-486d-955d-348383178022 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:13.599672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd554-bb94-486d-955d-348383178022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:13.639249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a31cd554-bb94-486d-955d-348383178022 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:13.641753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a31cd554-bb94-486d-955d-348383178022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:13.800685 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1025/2040] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:13.809335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:13.810276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:13.810276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:13.849790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:13.853325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:13.872930 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1016/2041] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:13.877161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd554-bb94-486d-955d-348383178022 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:13.890795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd554-bb94-486d-955d-348383178022 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:13.898785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:13.902124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:13.902124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:13.902124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:13.902124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:13.902704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:13.904508 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:13.918214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:13.922028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:13.931976 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1026/2042] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:11:13.933065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:13.947747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4f9b-fd45-47d0-9dfc-db08d1f249f1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:13.953715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:13.954604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:13.954992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:13.955523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:13.956507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:13.959378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:13.972388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e470e00-ff3b-41a8-903c-4f395b42fb33 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:13.998278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:13.999496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.000237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.000948 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] PATH_INFO: `/v3/projects/ed2d8394d6c84974aa1d459f8a758668` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.009036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2d8394d6c84974aa1d459f8a758668)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.015662 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.139083 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1027/2043] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:13 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:14.147632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.205128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2d8394d6c84974aa1d459f8a758668', 'request_id': 'req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:14.213199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2d8394d6c84974aa1d459f8a758668', 'request_id': 'req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:14.221492 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2d8394d6c84974aa1d459f8a758668', 'request_id': 'req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:14.242065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0d3-55f6-411d-91f1-087d48fcafcd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.321749 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] The token cache is being invalidate because project ed2d8394d6c84974aa1d459f8a758668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:14.322768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:14.325125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a509ce71-e907-46fc-a8aa-4e9f7abbefaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:14.326524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.327324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.327913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.328432 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.329252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.330292 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1017/2044] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:13 2022] DELETE /identity/v3/projects/ed2d8394d6c84974aa1d459f8a758668 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:14.332070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.341179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fccb4d-7440-4c0a-8132-fbab61888078 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.342109 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fccb4d-7440-4c0a-8132-fbab61888078 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.343164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fccb4d-7440-4c0a-8132-fbab61888078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.378585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c5fccb4d-7440-4c0a-8132-fbab61888078 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:14.381182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c5fccb4d-7440-4c0a-8132-fbab61888078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:14.393709 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1018/2045] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:14.409965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.410256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.410545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.441062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:14.443868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:14.458769 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1019/2046] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:14.468075 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1028/2047] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/users => generated 291 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:14.468425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.472717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.475739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:14.486167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ae130-385f-41be-b05c-1b3edd315e87 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.495442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.496487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.497201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.497665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.498316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a1388-372b-434c-a531-975b68b997e2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.498771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:14.499538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.506915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.507249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.508471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.508923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.509391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.510723 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.513139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.536803 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1020/2048] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:11:14.537076 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1029/2049] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:14.537238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.538205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.548327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2a50b-1a2a-4c4c-a334-b641189d148f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.550269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277c5c-2e2d-4855-a79c-eac53ec4c3b3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.557297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.558144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.558618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.559029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.560642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.563090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.567942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.569126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.569630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.570125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.571357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.576018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.597776 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1021/2050] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:14.603920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.624952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e29307-95de-4c4f-a7f2-b3fedad909ec admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.635089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.636312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.636824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.637648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] PATH_INFO: `/v3/projects/209dba9c7ab04f6b8b485660377a9485/users/0c4a554f56084a7fbf7d25627b16b69f/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.700127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] RBAC: Authorizing `identity:create_grant(project_id=209dba9c7ab04f6b8b485660377a9485, user_id=0c4a554f56084a7fbf7d25627b16b69f, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.716141 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1030/2051] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:14.720025 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.725456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da75878-dfb2-49e3-8304-1772ac67179c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.870634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb07e-62dd-4b47-99fd-6cd36ab861d2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.873425 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1022/2052] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:14 2022] PUT /identity/v3/projects/209dba9c7ab04f6b8b485660377a9485/users/0c4a554f56084a7fbf7d25627b16b69f/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:14.898753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.905935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.905935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.976832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:15.023688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] MFA Rules not processed for user `0c4a554f56084a7fbf7d25627b16b69f`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:15.025309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:15.059508 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.064491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.065091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.066796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.069511 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.075263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.250278 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1031/2053] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/users => generated 292 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:15.273411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.283435 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1023/2054] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:15.284719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.286237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:15.339929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d0f38-7b66-42e6-a866-5a9095680072 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.353618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.356225 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.357168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.357846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.363039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.363039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.388379 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1032/2055] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:15.389498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.394873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5c101-d094-4104-803b-6389ae5f8377 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.404326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.405180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.405816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.415826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.416132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79c7b9-889d-416f-838b-5f40a7bc8e8e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.418243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:15.432716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.436640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.437071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.437527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.438337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.443111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.450335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:15.546877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.549143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.572033 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1024/2056] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:15.574251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.612836 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1033/2057] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:15 2022] POST /identity/v3/projects => generated 379 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:15.618716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.622633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664f36d-bc7e-4ebd-b63c-7623bfb9a110 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.784425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99909dbe-2bff-4681-b809-28786bafa7a1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.794948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.796252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.797071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.797752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.799108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.803067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.803762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.811487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.811487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.811860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.812022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.812022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.825421 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1025/2058] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:15.826795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.842834 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4e4fe-f191-49a0-b42c-e836b66d19a6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:15.848730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:15.849480 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.849771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.850059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] PATH_INFO: `/v3/projects/ec0e8d209ae34edca0d7dc5dae71129a/users/9254fc77e32147fdb3a28422b8458039/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.873306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0e8d209ae34edca0d7dc5dae71129a, user_id=9254fc77e32147fdb3a28422b8458039, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:15.876525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54f26-dd16-4972-93ea-750853030647 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:15.965043 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1034/2059] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:15 2022] POST /identity/v3/users => generated 296 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:15.969620 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1026/2060] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:15 2022] PUT /identity/v3/projects/ec0e8d209ae34edca0d7dc5dae71129a/users/9254fc77e32147fdb3a28422b8458039/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:15.969896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:15.977858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:15.978380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:15.978927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:15.998650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f61-426c-4154-abbd-12ac223ead0d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.020347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:16.022768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:16.205670 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1027/2061] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:16.209560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.210657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.211178 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.212752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.213768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:16.217743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:16.223911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:16.226211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:16.256445 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1035/2062] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:16.267855 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:16.291052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816307-0bb5-40ab-b43c-f7ca8730acd6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.299425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.300891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.301608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.302793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] PATH_INFO: `/v3/projects/a7a6727a2f544c9db2653bd53ead1972/users/464923cd18c140de8eee170ca871aded/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.336022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a6727a2f544c9db2653bd53ead1972, user_id=464923cd18c140de8eee170ca871aded, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:16.343090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:16.372081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2f7b7-9d31-437d-9ed5-4e0ed3cf0afe None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.379228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.380537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.381154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.381796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.397270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:16.399073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:16.412396 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1028/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:16.453955 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1036/2064] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:16 2022] PUT /identity/v3/projects/a7a6727a2f544c9db2653bd53ead1972/users/464923cd18c140de8eee170ca871aded/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:16.459862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:16.475914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:16.484120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69fe-9170-4680-9996-d2dfb6a3eb48 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.617258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.618362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.619069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.619556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.620861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:16.625236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cba7c-7730-4b20-94fb-6b5af7c5cfce admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:16.665276 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1029/2065] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:16.669261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:16.673728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:16.705242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52e54-4b57-426c-8dd4-65a343496653 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.746402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.748506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.751715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.754114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] PATH_INFO: `/v3/projects/a7a6727a2f544c9db2653bd53ead1972/users/464923cd18c140de8eee170ca871aded/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.762557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a6727a2f544c9db2653bd53ead1972, user_id=464923cd18c140de8eee170ca871aded, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:16.769598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:16.805280 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1037/2066] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:16 2022] PUT /identity/v3/projects/a7a6727a2f544c9db2653bd53ead1972/users/464923cd18c140de8eee170ca871aded/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:16.808838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3aa3-5936-4128-b6fb-89a89e4f2ee8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.809817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3aa3-5936-4128-b6fb-89a89e4f2ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.810530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3aa3-5936-4128-b6fb-89a89e4f2ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.896117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-cd5a3aa3-5936-4128-b6fb-89a89e4f2ee8 None None] MFA Rules not processed for user `464923cd18c140de8eee170ca871aded`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:16.899696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cd5a3aa3-5936-4128-b6fb-89a89e4f2ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:17.223388 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1030/2067] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:17.238649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:17.242879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:17.334065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047013-0afd-424b-b8ee-1177b3210eb8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:17.334556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047013-0afd-424b-b8ee-1177b3210eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:17.335298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047013-0afd-424b-b8ee-1177b3210eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:17.393448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfdaaa-8626-49dd-9179-57526c4cb779 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:17.401257 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:17.404187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:17.404480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:17.404735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:17.405981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1b047013-0afd-424b-b8ee-1177b3210eb8 None None] MFA Rules not processed for user `19714f6672d345a38370effe826a3d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:17.408658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1b047013-0afd-424b-b8ee-1177b3210eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:17.428620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:17.430348 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:17.443636 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1038/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:17.537325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:17.548262 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1031/2069] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:18.232864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:18.243057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530dff2-5415-4b20-94fa-88a87c6ea9e7 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:18.248047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:18.248866 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:18.249169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:18.252519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:18.252519 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:18.391904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:18.393383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:18.416017 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1039/2070] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:18.884298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:19.269461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:19.272053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:19.272564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:19.388139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] MFA Rules not processed for user `9254fc77e32147fdb3a28422b8458039`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:19.390900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:19.608278 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1032/2071] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:19.670932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:19.693398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:19.726328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:19.743005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757d5e-441f-4e4d-b5c1-28a6b4cbe3ff service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:19.777517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:19.778286 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:19.778721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:19.779887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:19.851551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:19.857235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086d1fc-fbc9-4891-b7eb-4bca742ea9d1 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:19.917181 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1040/2072] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:20.014903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0af92-900d-409c-a653-4dea9d5d9ebd None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:20.032717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:20.032717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:20.032717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:20.032717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] PATH_INFO: `/v3/users/0c4a554f56084a7fbf7d25627b16b69f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:20.033407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4a554f56084a7fbf7d25627b16b69f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:20.043656 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:20.380276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4a554f56084a7fbf7d25627b16b69f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:20.382782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4a554f56084a7fbf7d25627b16b69f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:20.408624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4a554f56084a7fbf7d25627b16b69f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:20.500769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4a554f56084a7fbf7d25627b16b69f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:20.517990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4a554f56084a7fbf7d25627b16b69f'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:20.529580 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1033/2073] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:19 2022] DELETE /identity/v3/users/0c4a554f56084a7fbf7d25627b16b69f => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:20.534829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:20.535188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:20.536531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:20.561889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:20.563068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:20.640045 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1041/2074] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:20.655270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:20.665106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:20.761674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98750-4ec7-4b5a-96af-923156f70938 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:20.766312 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:20.767009 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:20.767293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:20.767569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:20.782715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:20.785645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:20.806770 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1034/2075] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:20.828592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:22.011806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:22.018840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:22.030685 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10048ff0-17d6-431d-ae31-fd149459746a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:22.038198 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:22.039272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:22.040613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:22.041130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] PATH_INFO: `/v3/projects/209dba9c7ab04f6b8b485660377a9485` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:22.043542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=209dba9c7ab04f6b8b485660377a9485)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:22.046076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:22.274512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209dba9c7ab04f6b8b485660377a9485', 'request_id': 'req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.281251 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209dba9c7ab04f6b8b485660377a9485', 'request_id': 'req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.399205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209dba9c7ab04f6b8b485660377a9485', 'request_id': 'req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.413683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] The token cache is being invalidate because project 209dba9c7ab04f6b8b485660377a9485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:22.413932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.414665 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b74b2fe4-1304-4f5b-8e8d-79a1226a8d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.421098 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1042/2076] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:22 2022] DELETE /identity/v3/projects/209dba9c7ab04f6b8b485660377a9485 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:22.424099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:22.436107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f451-7347-4331-8d7b-8ba558f4687a service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:22.489737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:22.490190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:22.491228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:22.526380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] MFA Rules not processed for user `9254fc77e32147fdb3a28422b8458039`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:22.528645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:22.549064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:22.550097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:22.550839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:22.552242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] PATH_INFO: `/v3/users/464923cd18c140de8eee170ca871aded` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:22.553804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=464923cd18c140de8eee170ca871aded)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:22.558846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:22.652079 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1043/2077] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:22.874905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464923cd18c140de8eee170ca871aded'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.875765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464923cd18c140de8eee170ca871aded'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:22.897781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464923cd18c140de8eee170ca871aded'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:23.021317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464923cd18c140de8eee170ca871aded'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:23.034223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464923cd18c140de8eee170ca871aded'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:23.042598 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1035/2078] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:22 2022] DELETE /identity/v3/users/464923cd18c140de8eee170ca871aded => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:23.137547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:23.774253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:23.780177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:23.806972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052038b-dd6f-452c-b443-e263b0e2aeb7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:23.945267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:23.946319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:23.946829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:23.947410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] PATH_INFO: `/v3/projects/a7a6727a2f544c9db2653bd53ead1972` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:23.956262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a6727a2f544c9db2653bd53ead1972)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:23.956262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:24.144696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a6727a2f544c9db2653bd53ead1972', 'request_id': 'req-32a38e9c-ea06-4588-80c5-d66b4a165bff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.169576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a6727a2f544c9db2653bd53ead1972', 'request_id': 'req-32a38e9c-ea06-4588-80c5-d66b4a165bff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.197301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:24.363507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:24.368141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:24.375366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a6727a2f544c9db2653bd53ead1972', 'request_id': 'req-32a38e9c-ea06-4588-80c5-d66b4a165bff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.387471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] The token cache is being invalidate because project a7a6727a2f544c9db2653bd53ead1972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:24.388461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.390297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-32a38e9c-ea06-4588-80c5-d66b4a165bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.393422 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1044/2079] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:23 2022] DELETE /identity/v3/projects/a7a6727a2f544c9db2653bd53ead1972 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:24.411103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5ae6-950c-4aac-adfe-bb1df1d1452d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:24.411103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5ae6-950c-4aac-adfe-bb1df1d1452d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:24.411103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5ae6-950c-4aac-adfe-bb1df1d1452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:24.433649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dacaa-efe1-435c-a482-e12b8bf8a2ab admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:24.545524 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-464e5ae6-950c-4aac-adfe-bb1df1d1452d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:24.547739 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-464e5ae6-950c-4aac-adfe-bb1df1d1452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:24.647574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:24.649030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:24.649526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:24.649900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] PATH_INFO: `/v3/users/dba0acf6b5cd4069bb149b9eca4aedb7` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:24.651798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba0acf6b5cd4069bb149b9eca4aedb7)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:24.656477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:24.775376 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1045/2080] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:24.786794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:24.787441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:24.787891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:24.835480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:24.837188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:24.859981 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1046/2081] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:24.867276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:24.882350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972608-59ef-4e96-b55f-7fff68203a0a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:24.890504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:24.899769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:24.917276 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1047/2082] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:11:24.920494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:24.945964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332646-b9c1-44bf-a9b5-d1ffe672f796 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:24.956444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:24.957642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:24.958171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:24.959104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:24.960687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:24.965699 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:24.978079 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba0acf6b5cd4069bb149b9eca4aedb7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:24.980264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba0acf6b5cd4069bb149b9eca4aedb7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:25.012517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba0acf6b5cd4069bb149b9eca4aedb7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:25.108963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba0acf6b5cd4069bb149b9eca4aedb7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:25.112774 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1048/2083] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:24 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:25.119231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:25.123368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:25.139464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-4898fe1f-7bbb-4ef8-b86e-8cf4d30098c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba0acf6b5cd4069bb149b9eca4aedb7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:25.166679 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1036/2084] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:24 2022] DELETE /identity/v3/users/dba0acf6b5cd4069bb149b9eca4aedb7 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:25.172984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.173360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.174223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.198940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:25.200224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:25.219366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6791-9a9a-4205-8c97-6d9599fbab48 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:25.288232 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1037/2085] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:25.300393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:25.303870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:25.335328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:25.336207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.336643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.337070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.339410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:25.342728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:25.402411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c62a8-aa2e-469a-a51e-9d4b2e31c439 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:25.407306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:25.408324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.408790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.413149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.424611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:25.428617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:25.442527 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1038/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:25.468900 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1049/2087] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:25 2022] POST /identity/v3/users => generated 286 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:25.470300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:25.481914 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5bee9-bdc9-4ade-8b8a-4fc7e0545446 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:25.492566 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:25.495859 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.497226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.497226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.497953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:25.500970 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:25.513651 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1039/2088] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:25.519558 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:25.539729 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3c4e0-1e33-4618-87cb-ec1b02581c87 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:25.550004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:25.552863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.553671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.555954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.557673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:25.561173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dec7e-55f3-444b-be24-8c91ab446dc5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:25.588732 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1050/2089] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:25.591276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:25.605599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f4a-e5cd-4673-8e6e-6b1298eb949b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:25.612524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:25.613365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.613695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.613979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] PATH_INFO: `/v3/projects/a0dd3a3ba566479d94d5438d890da279/users/7773de8f0ee743c9881d441e8e9d8fc2/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.635740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dd3a3ba566479d94d5438d890da279, user_id=7773de8f0ee743c9881d441e8e9d8fc2, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:25.638799 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:25.758128 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1040/2090] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:25 2022] PUT /identity/v3/projects/a0dd3a3ba566479d94d5438d890da279/users/7773de8f0ee743c9881d441e8e9d8fc2/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:25.765260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317033d5-034f-4f75-a347-108f5a722085 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:25.765547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317033d5-034f-4f75-a347-108f5a722085 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:25.766140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317033d5-034f-4f75-a347-108f5a722085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:25.825162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-317033d5-034f-4f75-a347-108f5a722085 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:25.826155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:25.828564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-317033d5-034f-4f75-a347-108f5a722085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:25.931673 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1051/2091] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:25.943994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:26.012025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad2a6c-2d0e-46a2-bab5-ad1f17c69aa9 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:26.016136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:26.016933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:26.017222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:26.017503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:26.028773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:26.031765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:26.054038 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:26.055847 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1041/2092] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:26.422477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317033d5-034f-4f75-a347-108f5a722085 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:26.424683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-317033d5-034f-4f75-a347-108f5a722085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:26.449033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317033d5-034f-4f75-a347-108f5a722085 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:26.463430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:26.465766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:26.465766 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:26.466364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] PATH_INFO: `/v3/projects/61f624796bac4252ad0e0b5d117284f1` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:26.470861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f624796bac4252ad0e0b5d117284f1)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:26.487093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:26.691942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f624796bac4252ad0e0b5d117284f1', 'request_id': 'req-bbe90723-be83-4a74-a52d-94a9a415118d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:26.708124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f624796bac4252ad0e0b5d117284f1', 'request_id': 'req-bbe90723-be83-4a74-a52d-94a9a415118d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:26.822764 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f624796bac4252ad0e0b5d117284f1', 'request_id': 'req-bbe90723-be83-4a74-a52d-94a9a415118d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:26.838612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] The token cache is being invalidate because project 61f624796bac4252ad0e0b5d117284f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:26.839863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:26.841506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bbe90723-be83-4a74-a52d-94a9a415118d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:26.845115 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1052/2093] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:26 2022] DELETE /identity/v3/projects/61f624796bac4252ad0e0b5d117284f1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:26.849658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:26.887189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5990efd-79b0-42a4-9394-a7a51d3632e8 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:26.994418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:26.995252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:26.995756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:26.996938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] PATH_INFO: `/v3/users/b3125a76e7824f2b8c7cbe81c7837495` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:26.998376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3125a76e7824f2b8c7cbe81c7837495)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:27.002246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:27.343060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3125a76e7824f2b8c7cbe81c7837495'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:27.344450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3125a76e7824f2b8c7cbe81c7837495'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:27.380660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3125a76e7824f2b8c7cbe81c7837495'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:27.533907 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3125a76e7824f2b8c7cbe81c7837495'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:27.549081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3125a76e7824f2b8c7cbe81c7837495'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:27.568951 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1042/2094] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:26 2022] DELETE /identity/v3/users/b3125a76e7824f2b8c7cbe81c7837495 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:28.046885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:28.047613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:28.047613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:28.108158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] MFA Rules not processed for user `7773de8f0ee743c9881d441e8e9d8fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:28.112383 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:28.253906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:28.256167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:28.303358 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1053/2095] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:28.307228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:28.313579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:28.442326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e94a-ff03-4a59-b17d-2ae2c2440533 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:28.443862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4627-2d77-4a27-8446-a6a1838aa655 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:28.449050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:28.449527 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:28.449688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:28.450188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:28.450467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:28.450629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:28.451090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:28.451203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:28.451617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] PATH_INFO: `/v3/projects/2c4d498132c04cec9b72f18a6e23ef57` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:28.453109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:28.453710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4d498132c04cec9b72f18a6e23ef57)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:28.458044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:28.565253 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1054/2096] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:28 2022] POST /identity/v3/projects => generated 371 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:28.567219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:28.649246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08050e-0563-4477-8d15-196e3fbb7ead admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:28.656857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:28.657471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:28.657853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:28.658270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:28.659132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:28.661935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:28.683643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4d498132c04cec9b72f18a6e23ef57', 'request_id': 'req-2b643003-e550-4774-9c7d-29ffddd589af'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:28.689800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4d498132c04cec9b72f18a6e23ef57', 'request_id': 'req-2b643003-e550-4774-9c7d-29ffddd589af'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:28.696954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4d498132c04cec9b72f18a6e23ef57', 'request_id': 'req-2b643003-e550-4774-9c7d-29ffddd589af'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:28.797215 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1055/2097] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:28 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:28.801959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:28.816299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e67e-1874-4145-a3ef-f11a7de6f501 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:28.845992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] The token cache is being invalidate because project 2c4d498132c04cec9b72f18a6e23ef57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:28.846503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:28.848850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:28.857555 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1043/2098] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:28 2022] DELETE /identity/v3/projects/2c4d498132c04cec9b72f18a6e23ef57 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:28.859562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:28.865766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:29.000458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:29.005210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:29.006101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:29.007392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:29.007913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:29.012646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:29.014543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:29.024093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b643003-e550-4774-9c7d-29ffddd589af admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:29.031338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:29.032412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:29.032898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:29.034426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] PATH_INFO: `/v3/users/77238492505b434b9fb73491aae13b72` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:29.035864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77238492505b434b9fb73491aae13b72)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:29.042760 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1056/2099] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:29.043198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:29.046751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:29.050592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:29.107086 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c0e4-9f24-4bd5-b6cf-8504ced9512f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:29.112661 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:29.113410 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:29.113714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:29.114196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] PATH_INFO: `/v3/projects/93aecb6a030847cd868359b1201b5121/users/7424867fb1f94222b60e82d6df82a253/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:29.134357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93aecb6a030847cd868359b1201b5121, user_id=7424867fb1f94222b60e82d6df82a253, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:29.137211 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:29.258573 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1057/2100] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:29 2022] PUT /identity/v3/projects/93aecb6a030847cd868359b1201b5121/users/7424867fb1f94222b60e82d6df82a253/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:29.263629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:29.316668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77238492505b434b9fb73491aae13b72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:29.318257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77238492505b434b9fb73491aae13b72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:29.345431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77238492505b434b9fb73491aae13b72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:29.372506 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d7ddd-b9ba-4085-817d-75a3de5b065d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:29.379181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:29.380602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:29.380895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:29.381946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:29.381946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:29.383744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:29.406515 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1058/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:11:29.412269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:29.415324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:29.420670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77238492505b434b9fb73491aae13b72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:29.435033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77238492505b434b9fb73491aae13b72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:29.444673 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fee7-78ca-40b7-bdda-a28f4389b6ba admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:29.445656 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1044/2102] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:28 2022] DELETE /identity/v3/users/77238492505b434b9fb73491aae13b72 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:29.575490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:29.577210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:29.577927 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:29.578718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] PATH_INFO: `/v3/projects/93aecb6a030847cd868359b1201b5121/users/7424867fb1f94222b60e82d6df82a253/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:29.584629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93aecb6a030847cd868359b1201b5121, user_id=7424867fb1f94222b60e82d6df82a253, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:29.592687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472603de-9729-49c3-b049-1445f0cea82f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:29.620411 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1059/2103] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:29 2022] PUT /identity/v3/projects/93aecb6a030847cd868359b1201b5121/users/7424867fb1f94222b60e82d6df82a253/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:29.878422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:30.533321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:30.537053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:30.558741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5c260-02cd-4ad1-a138-cf88d5efa1f4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:30.785978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:30.788761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:30.789454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:30.790142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] PATH_INFO: `/v3/projects/e552720985df4365b6e97afc139ed397` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:30.792537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e552720985df4365b6e97afc139ed397)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:30.797323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:30.981116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e552720985df4365b6e97afc139ed397', 'request_id': 'req-0c908362-4e8c-4bca-837c-1a6109973004'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:30.992284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e552720985df4365b6e97afc139ed397', 'request_id': 'req-0c908362-4e8c-4bca-837c-1a6109973004'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:31.003672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e552720985df4365b6e97afc139ed397', 'request_id': 'req-0c908362-4e8c-4bca-837c-1a6109973004'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:31.134516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] The token cache is being invalidate because project e552720985df4365b6e97afc139ed397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:31.134832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:31.135532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0c908362-4e8c-4bca-837c-1a6109973004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:31.140762 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1045/2104] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:30 2022] DELETE /identity/v3/projects/e552720985df4365b6e97afc139ed397 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:31.142833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:31.143672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:31.144161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:31.169890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:31.183371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:31.339973 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1060/2105] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:31.342405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:31.344394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:31.347438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:31.451359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:31.463693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:31.474906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:31.490314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:31.498754 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1046/2106] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:31.516142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7dfa6-b7be-4afb-b49c-12e149c940fb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:31.525430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:31.526469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:31.526954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:31.527440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:31.534493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:31.535005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:31.538262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:31.566555 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1061/2107] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:31.573368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:31.596487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4499709-2b7c-49d7-88d2-b35d7b28b966 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:31.607409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:31.608639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:31.609276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:31.609666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:31.610898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:31.614568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:31.747424 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1047/2108] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:31 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:31.754846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:31.774067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637714d-5e6e-4095-85df-4cf42ad55789 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:31.901126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:31.901832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:31.902272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:31.902635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:31.903358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:31.906197 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:32.034507 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1062/2109] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:31 2022] POST /identity/v3/users => generated 286 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:32.040929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:32.063823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41786e-40d4-488d-ac2c-5de1330d5c45 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:32.074792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:32.075674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:32.076175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:32.076887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:32.078095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:32.081125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:32.099172 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1048/2110] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:32.112143 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:32.149935 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8aa17-c95a-4f4e-9327-7e9f5e305cc1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:32.157594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:32.158387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:32.161177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:32.161733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:32.168709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:32.171468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347bf0b-f30f-4c61-afee-ece69383c51a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:32.199388 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1063/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:32.206647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:32.225298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a598-4913-407d-9a37-1fc3b2325b13 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:32.232965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:32.234008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:32.234599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:32.235058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] PATH_INFO: `/v3/projects/818a80823efd48c1923d03f9769f4376/users/cfd273c23f3d4329a7cdbba2ca113523/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:32.260155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818a80823efd48c1923d03f9769f4376, user_id=cfd273c23f3d4329a7cdbba2ca113523, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:32.263908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:32.446495 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1049/2112] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:32 2022] PUT /identity/v3/projects/818a80823efd48c1923d03f9769f4376/users/cfd273c23f3d4329a7cdbba2ca113523/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:32.464952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a2a8-d44d-4c05-9984-25bd2d08dbee None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:32.465530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a2a8-d44d-4c05-9984-25bd2d08dbee None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:32.466443 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a2a8-d44d-4c05-9984-25bd2d08dbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:32.542444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f976a2a8-d44d-4c05-9984-25bd2d08dbee None None] MFA Rules not processed for user `cfd273c23f3d4329a7cdbba2ca113523`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:32.544573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f976a2a8-d44d-4c05-9984-25bd2d08dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:32.711847 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1064/2113] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:32.734883 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:32.737683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:32.884844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b45c8-8c76-4420-9cb7-8e843ad10174 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:32.892371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:32.893597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:32.894198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:32.895380 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:32.909745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:32.911748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fdd40-10cf-43a2-ab29-9a1bf9e6a453 service glance] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:32.926684 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1050/2114] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:33.038008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:33.102656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:33.106791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:33.106791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:33.188517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] MFA Rules not processed for user `7424867fb1f94222b60e82d6df82a253`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:33.192521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:33.268636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:33.402821 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1065/2115] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:33.415406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:33.416034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:33.416471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:33.448226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] MFA Rules not processed for user `7773de8f0ee743c9881d441e8e9d8fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:33.450687 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:33.598812 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1051/2116] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:33.605676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:33.608780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:33.725100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481837a3-7a8c-4de5-a97a-a3945bae59b4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:33.730684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:33.731455 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:33.731676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:33.732018 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:33.733901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:33.746180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:33.748151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d96eb8-a272-4c5d-a5b4-24d0d0a36351 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:33.770091 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1066/2117] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:34.512073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:35.543036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:36.223822 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:37.696373 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:39.136892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:39.523854 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:41.009885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:41.203102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:42.089374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:42.321456 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:42.994974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:43.814033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:43.949090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:44.808242 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:44.842437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:45.130526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:45.157008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:45.505578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:45.928675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.180322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.520116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.546407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.677298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.787818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:46.928418 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:47.099997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:47.459989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:47.706261 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:48.044212 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:48.278346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:48.779847 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:49.564272 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:49.864411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:50.096877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:50.609363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:50.671975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.032207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.415505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.648081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.717371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.862502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:51.969565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:52.252410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:52.910518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.145529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.429522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.496927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:53.498489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:53.547264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.604066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a058-68c7-47c1-a0c4-0bf27e32d355 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:53.609784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:53.610138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.610871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:53.611361 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:53.611742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] PATH_INFO: `/v3/users/fad56d680ba2475caf240311bf625e99` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:53.619814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad56d680ba2475caf240311bf625e99)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:53.619814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:53.929389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:53.973573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad56d680ba2475caf240311bf625e99'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:53.974550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad56d680ba2475caf240311bf625e99'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:53.995374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad56d680ba2475caf240311bf625e99'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:54.089757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:54.091044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad56d680ba2475caf240311bf625e99'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:54.102837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad56d680ba2475caf240311bf625e99'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:54.113544 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1052/2118] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:53 2022] DELETE /identity/v3/users/fad56d680ba2475caf240311bf625e99 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:54.118894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1571f-f973-4a0c-ad7f-9935ab837966 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:54.119206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1571f-f973-4a0c-ad7f-9935ab837966 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:54.119485 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1571f-f973-4a0c-ad7f-9935ab837966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:54.149892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-85e1571f-f973-4a0c-ad7f-9935ab837966 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:54.149892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-85e1571f-f973-4a0c-ad7f-9935ab837966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:54.215358 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1067/2119] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:54.226861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:54.228587 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:54.294106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:54.302504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4911d-3aab-4545-a5f7-8739cddc08df admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:54.306644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:54.316423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:54.316423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:54.316423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:54.320063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:54.321540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:54.333579 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1053/2120] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:11:54.415034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:54.602171 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:54.770198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:54.781578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:54.782075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:54.782588 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:54.839801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:54.844053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:54.945614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:54.947472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:54.974616 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1068/2121] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:54.986893 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605eb2-aac2-4c12-8453-3025ca938944 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:54.992719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:54.993387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:54.993678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:54.993956 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] PATH_INFO: `/v3/projects/7b63baebedae48bc97e663daf54fb710` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:54.995911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b63baebedae48bc97e663daf54fb710)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:55.004300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:55.053186 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:55.054847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:55.152085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea071d-313b-4e52-aea6-bd4bdec3f794 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:55.157047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:55.175212 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:55.180587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.180587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.180587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.187757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b63baebedae48bc97e663daf54fb710', 'request_id': 'req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:55.193477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:55.196532 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b63baebedae48bc97e663daf54fb710', 'request_id': 'req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:55.196792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562be5-d0c2-49d7-8487-5473d3447d33 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:55.204436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b63baebedae48bc97e663daf54fb710', 'request_id': 'req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:55.213138 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1069/2122] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:11:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 30 01:11:55.297335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:55.306274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] The token cache is being invalidate because project 7b63baebedae48bc97e663daf54fb710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:55.306759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:55.308072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d0c5ced3-b07a-4b7f-bbb8-48bcccbc5165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:55.310899 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1054/2123] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:54 2022] DELETE /identity/v3/projects/7b63baebedae48bc97e663daf54fb710 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:55.317796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.318806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.319609 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.357405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:55.359600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:55.478042 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1070/2124] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:55.488838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.489158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.489488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.515070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:55.516886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:55.530973 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1055/2125] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:55.533136 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:55.552321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:55.570269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c1701-a651-4218-88e2-bda37acb8af1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:55.607279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:55.608290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.608674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.609249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.609963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:55.610307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:55.612085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:55.641979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:55.646936 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1071/2126] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:55.655295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3f4bc-4537-4765-b8dc-6551c2770454 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:55.664754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:55.665401 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.665755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.666123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.666705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:55.671062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:55.812746 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1056/2127] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:55 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:55.817851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:55.829422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2bdd7-07ee-4005-b1fa-92569dd596fd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:55.926818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:55.928026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:55.928452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:55.928950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:55.930203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:55.933976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:56.054907 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1072/2128] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:55 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:56.058736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:56.074733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0bf39-0ec8-40de-9b78-424edf1af559 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:56.074963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:56.084362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:56.085898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:56.086460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:56.086919 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:56.088327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:56.090245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:56.104256 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1057/2129] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:56.109832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:56.148659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ddcd-8aad-423e-91bd-40d77fabb859 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:56.161612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:56.162582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:56.163055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:56.163331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:56.164051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:56.166551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371502a7-60f2-4b85-8906-69a6be937e70 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:56.173667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:56.185812 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1073/2130] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:11:56.192652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:56.215429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd398-5842-4abe-9c86-6f875b35ab0c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:56.236415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:56.244257 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:56.247056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:56.248512 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] PATH_INFO: `/v3/projects/91233450c8174a8ca3ceef9fa27f2f7d/users/fca56015522c41d2b121d3a93e8a39e9/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:56.305573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:56.328254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91233450c8174a8ca3ceef9fa27f2f7d, user_id=fca56015522c41d2b121d3a93e8a39e9, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:56.332423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:56.377813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:56.551800 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1058/2131] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:11:56 2022] PUT /identity/v3/projects/91233450c8174a8ca3ceef9fa27f2f7d/users/fca56015522c41d2b121d3a93e8a39e9/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:56.558401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:56.560046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:56.560691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:56.607521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] MFA Rules not processed for user `fca56015522c41d2b121d3a93e8a39e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:56.609205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:56.808096 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1074/2132] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:56.834217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:56.839477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:57.023743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f336b-2f99-4483-9739-ecd24d0720c0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:57.029855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:57.031070 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:57.031778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:57.032717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:57.050037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:57.052209 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145ce9e-4a34-4f94-abcb-e29499f6d703 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:57.067574 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1059/2133] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:57.112488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:57.170534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:57.234449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdfce8-4a8c-4785-9719-d9744eed07ca None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:57.246322 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:57.247337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:57.247689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:57.248079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] PATH_INFO: `/v3/users/fca56015522c41d2b121d3a93e8a39e9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:57.252905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca56015522c41d2b121d3a93e8a39e9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:57.255902 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:57.544577 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca56015522c41d2b121d3a93e8a39e9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:57.545517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca56015522c41d2b121d3a93e8a39e9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:57.561379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca56015522c41d2b121d3a93e8a39e9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:57.562357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca56015522c41d2b121d3a93e8a39e9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:57.592378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca56015522c41d2b121d3a93e8a39e9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:57.713315 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1075/2134] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:11:57 2022] DELETE /identity/v3/users/fca56015522c41d2b121d3a93e8a39e9 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:11:57.718469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:57.719134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:57.719225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:57.746429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:57.757387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:57.832608 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1060/2135] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:57.842864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:57.844574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:57.932569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c3425-61e3-45d0-acb0-70bc8e331b9f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:57.936470 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:57.937023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:57.937340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:57.937616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:57.948582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:57.949989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699a9d1-3224-4170-b6a9-b0191577a162 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:57.962365 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1076/2136] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:58.693171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:58.695112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:58.723869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57397a9b-609c-4d48-9ce4-7b7d160b857c None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:58.732220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:58.733425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:58.734002 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:58.734715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] PATH_INFO: `/v3/projects/91233450c8174a8ca3ceef9fa27f2f7d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:58.737581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91233450c8174a8ca3ceef9fa27f2f7d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:58.741536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:58.906899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:58.926187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91233450c8174a8ca3ceef9fa27f2f7d', 'request_id': 'req-f0019cbd-211b-4763-b8d7-06b122ed3224'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:58.935732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91233450c8174a8ca3ceef9fa27f2f7d', 'request_id': 'req-f0019cbd-211b-4763-b8d7-06b122ed3224'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:58.955861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91233450c8174a8ca3ceef9fa27f2f7d', 'request_id': 'req-f0019cbd-211b-4763-b8d7-06b122ed3224'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:59.072840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] The token cache is being invalidate because project 91233450c8174a8ca3ceef9fa27f2f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:11:59.073256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:59.074005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f0019cbd-211b-4763-b8d7-06b122ed3224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:11:59.079725 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1061/2137] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:11:58 2022] DELETE /identity/v3/projects/91233450c8174a8ca3ceef9fa27f2f7d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:11:59.093137 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.094265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.094829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.132167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:59.133909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:59.222052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:59.287632 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1077/2138] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:11:59.299144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.302329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.302329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.343229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:11:59.345413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:59.367142 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1062/2139] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:11:59.374399 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:59.385437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca57c6-5c95-4ecc-862d-42c977cfc17e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:59.391460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:59.392654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.393448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.394037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.395994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:11:59.396861 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:59.402201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:59.436191 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1078/2140] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:11:59.444925 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:59.470843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993cc99-af1f-428f-b9f1-032857d9c175 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:59.487095 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:59.489617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.490127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.491384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.493682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:59.500679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:59.531035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:59.535528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:59.635414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e31c-fe75-4345-81c3-c30febcc3079 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:59.639702 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1063/2141] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:11:59 2022] POST /identity/v3/projects => generated 375 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:11:59.646157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:59.754065 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:59.754811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.755201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.756208 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.758553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:59.797483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e186862-2b91-4097-a7df-89f65c9f87f3 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:59.806388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:59.807628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.808184 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.808968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:59.810259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:59.814170 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:59.878761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:59.883972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:59.905340 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1079/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:59.964872 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1064/2143] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:11:59 2022] POST /identity/v3/users => generated 288 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:11:59.965631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:59.987230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb5db-0643-4b87-a625-2022b43fe493 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:59.996156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:59.997970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:59.998815 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:59.999862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:00.001414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:00.005051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:00.051077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:00.059327 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1080/2144] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:12:00.071536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d37a6d-2db7-461d-8e30-54db7a7fd57a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:00.080794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:00.082421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:00.082976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:00.083518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:00.085127 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:00.088602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7242fd7-7145-4904-8cf7-0fbb8d1acc10 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:00.113676 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1065/2145] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:00.120112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:00.163452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aad9c-1f03-45bd-bd0d-452c9d0f34a9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:00.165716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:00.166895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:00.167248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:00.167620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] PATH_INFO: `/v3/projects/dc0d0c23838943198fb44e362e175548/users/b59ebba856cc4b008ea86265218907f6/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:00.196634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0d0c23838943198fb44e362e175548, user_id=b59ebba856cc4b008ea86265218907f6, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:00.199887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:00.307192 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1081/2146] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:00 2022] PUT /identity/v3/projects/dc0d0c23838943198fb44e362e175548/users/b59ebba856cc4b008ea86265218907f6/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:00.322913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:00.323261 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:00.323605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:00.363735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] MFA Rules not processed for user `b59ebba856cc4b008ea86265218907f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:00.365727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:00.475029 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1066/2147] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:00.548462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:00.551405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:00.562970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:00.636736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27401f-6e0b-4d66-aec8-dce4ce7f4007 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:00.641704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:00.642366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:00.642621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:00.642975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:00.655032 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:00.659717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de2b9-8734-4b6e-9c27-c0913af49f14 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:00.687645 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1082/2148] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:10.472436 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:10.888580 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=24 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:16.666376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:16.852119 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49795b7a-04a5-4b20-8505-73bd7dd5ce8b None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:16.858984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:16.859813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:16.860180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:16.860459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] PATH_INFO: `/v3/users/b59ebba856cc4b008ea86265218907f6` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:16.866687 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59ebba856cc4b008ea86265218907f6)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:16.870199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:17.225518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59ebba856cc4b008ea86265218907f6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:17.227686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59ebba856cc4b008ea86265218907f6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:17.277628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59ebba856cc4b008ea86265218907f6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:17.435649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59ebba856cc4b008ea86265218907f6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:17.464332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59ebba856cc4b008ea86265218907f6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:17.464332 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1067/2149] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:16 2022] DELETE /identity/v3/users/b59ebba856cc4b008ea86265218907f6 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:17.468000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:17.468972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:17.469429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:17.510572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:17.531422 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:17.689229 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1083/2150] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:17.700714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:17.703454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:17.852540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef326-052e-4a64-9407-9686380a001a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:17.865966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:17.867053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:17.867681 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:17.870033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:17.900786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:17.908761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe01de-d389-4761-91e5-c71d9dc84405 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:17.936265 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1068/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:18.617549 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:18.620552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:18.687296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3fe72-34be-46e9-a685-d9043978219d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:18.702883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:18.710292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:18.710292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:18.710292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] PATH_INFO: `/v3/projects/dc0d0c23838943198fb44e362e175548` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:18.710292 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0d0c23838943198fb44e362e175548)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:18.713607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:18.908802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0d0c23838943198fb44e362e175548', 'request_id': 'req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:18.920400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0d0c23838943198fb44e362e175548', 'request_id': 'req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:18.988868 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0d0c23838943198fb44e362e175548', 'request_id': 'req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:19.015877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] The token cache is being invalidate because project dc0d0c23838943198fb44e362e175548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:19.016077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:19.019306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:19.021532 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1084/2152] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:18 2022] DELETE /identity/v3/projects/dc0d0c23838943198fb44e362e175548 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:20.635122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:20.636081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:20.636081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:20.676348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:20.679561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:20.713399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:20.805783 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1069/2153] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:20.835142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:20.838414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:21.020589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5dd3-77c1-4157-86c8-48f6a6951cd9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:21.036270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:21.038419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:21.039011 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:21.039504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:21.054589 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:21.059672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e171de-8a63-40f6-921e-cb1ca92a0d11 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:21.079935 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1085/2154] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:12:24.135366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:24.137134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:24.159912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3198f-ec88-41e3-9b7c-73f416715b45 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:24.172722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:24.174407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:24.174913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:24.175402 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] PATH_INFO: `/v3/users/cfd273c23f3d4329a7cdbba2ca113523` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:24.177888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd273c23f3d4329a7cdbba2ca113523)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:24.181838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:24.531378 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd273c23f3d4329a7cdbba2ca113523'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:24.532148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd273c23f3d4329a7cdbba2ca113523'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:24.558323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd273c23f3d4329a7cdbba2ca113523'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:24.646614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd273c23f3d4329a7cdbba2ca113523'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:24.656798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd273c23f3d4329a7cdbba2ca113523'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:24.664962 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1070/2155] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:24 2022] DELETE /identity/v3/users/cfd273c23f3d4329a7cdbba2ca113523 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:24.670385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:24.671311 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:24.672340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:24.711639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:24.713898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:24.719773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:24.849992 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1086/2156] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:24.868046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:24.871083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:25.009479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ac7ae-ebac-47e4-b7b7-707d5ed6a648 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:25.018929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:25.020005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:25.020574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:25.020991 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:25.038962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:25.041674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ab008-cf01-4a41-ae3d-2df5b8fba106 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:25.062453 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1071/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:25.976283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:25.978704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:26.000318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e5f87-7ee5-45a7-afe6-a1141b0be8b2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:26.010497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:26.011462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.012001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.012501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] PATH_INFO: `/v3/projects/818a80823efd48c1923d03f9769f4376` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.014736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818a80823efd48c1923d03f9769f4376)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:26.019428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:26.171863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818a80823efd48c1923d03f9769f4376', 'request_id': 'req-06a7c674-717e-4147-b459-0bf93034b7d1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:26.179972 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818a80823efd48c1923d03f9769f4376', 'request_id': 'req-06a7c674-717e-4147-b459-0bf93034b7d1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:26.280611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818a80823efd48c1923d03f9769f4376', 'request_id': 'req-06a7c674-717e-4147-b459-0bf93034b7d1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:26.311937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] The token cache is being invalidate because project 818a80823efd48c1923d03f9769f4376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:26.312424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:26.313610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06a7c674-717e-4147-b459-0bf93034b7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:26.316851 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1087/2158] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:25 2022] DELETE /identity/v3/projects/818a80823efd48c1923d03f9769f4376 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:26.327006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.327863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.328408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.365024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:26.366284 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:26.420743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:26.464658 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1072/2159] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:26.475077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.475428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.475687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.500535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:26.503754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:26.517526 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1088/2160] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:26.522348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:26.533140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36bd93-2ab9-4914-a482-bdc66e8c13bf None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:26.538542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:26.539198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.542609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.542609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.542609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:12:26.542609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:26.542609 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:26.557061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1073/2161] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:12:26.563336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:26.574249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315076f-3cf4-4079-8e12-4346af0d0edf None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:26.579653 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:26.580329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.580614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.580979 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.581666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:26.583692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:26.710772 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1089/2162] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:12:26 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:26.715798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:26.732725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93605a-8fd1-4368-87ba-9d2db5c05361 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:26.801345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:26.801986 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.802270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.802553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.803231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:26.805285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:26.927494 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1074/2163] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:12:26 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:12:26.932838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:26.949002 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c210-6714-4971-94de-d74ae8897582 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:26.954333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:26.955465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:26.955792 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:26.956067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:26.956745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:26.958400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:26.974213 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1090/2164] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:26.979494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:26.994063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5fb89-3b7c-49d0-a3d3-bef2520aa779 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:27.007655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:27.007655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.007655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.007655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.008304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:27.018592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a840-ae89-4a1b-83af-23ce12e7b506 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:27.036148 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1075/2165] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:27.040636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:27.044611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.044889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.045224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.056610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a1e8-6d92-40de-b86a-833d00a16102 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:27.062941 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:27.063896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.064418 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.064971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] PATH_INFO: `/v3/projects/77968fc2b85349b2a625ea9bf25cd026/users/efbfd82026f14adb91bead1b95154448/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.067287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:27.068796 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.079115 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1076/2166] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:27.088683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77968fc2b85349b2a625ea9bf25cd026, user_id=efbfd82026f14adb91bead1b95154448, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:27.091914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4f377-0544-4373-bde0-9427b3ced283 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:27.103955 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:27.105440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.172364 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1091/2167] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:27 2022] PUT /identity/v3/projects/77968fc2b85349b2a625ea9bf25cd026/users/efbfd82026f14adb91bead1b95154448/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:27.190019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.190492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.191205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.214352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:27.215900 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.222462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd30bc-8492-4c71-91d4-d4237b59fb87 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:27.252170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:27.264641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:27.296719 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1092/2168] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:27.302742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:27.303442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.303779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.304071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.307856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:27.309276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.314839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:27.316811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a431-1244-45e2-b8dc-d2a4c54eabc3 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:27.330900 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1077/2169] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:12:27.411884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f950806-6601-496c-a85b-898eeec3ecd3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:27.420068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:27.420759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.421096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.421447 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.455568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:27.457342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:27.470250 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1093/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:12:27.542523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.543548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.544364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.578603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:27.582224 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.652386 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1078/2171] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:27.670112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:27.671716 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:27.776733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f212-6aee-4788-84cb-714cfb3407b0 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:27.782160 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:27.782794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:27.783109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:27.783388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:27.792591 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:27.794201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10e98e-b35e-4a93-8cc9-7c3dd9f3b224 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:27.803960 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1094/2172] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:12:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:12:32.589161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:32.590672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:32.610322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dd7df-e808-4b55-bbc7-14f1d699bf71 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:32.617767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:32.618947 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:32.619925 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:32.621237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] PATH_INFO: `/v3/users/19714f6672d345a38370effe826a3d2a` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:32.622857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19714f6672d345a38370effe826a3d2a)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:32.625244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:32.691888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:32.692450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:32.692890 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:32.740139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:32.745515 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:32.758345 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1095/2173] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:32.901802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19714f6672d345a38370effe826a3d2a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:32.904507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19714f6672d345a38370effe826a3d2a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:32.949292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19714f6672d345a38370effe826a3d2a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:33.045361 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:33.048484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19714f6672d345a38370effe826a3d2a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:33.052177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:33.075021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0f863-0f3d-4565-9af8-b8e5ad4b02cf None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:33.081690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19714f6672d345a38370effe826a3d2a'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:33.083145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:33.083836 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:33.091313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:33.091313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:33.091313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:33.095093 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1079/2174] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:32 2022] DELETE /identity/v3/users/19714f6672d345a38370effe826a3d2a => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:12:33.104565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:33.106334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:33.124008 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1096/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:12:33.679654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:33.681857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:33.777315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:33.834304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114195-15fa-4321-8d74-e26d1395cb6e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:33.840407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:33.840677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:33.840827 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:33.841233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] PATH_INFO: `/v3/projects/b3ac92b862ae4d988409b01c3adfef75` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:33.842759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ac92b862ae4d988409b01c3adfef75)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:33.844840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:34.006530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ac92b862ae4d988409b01c3adfef75', 'request_id': 'req-e4162d58-c295-4ebb-ae35-e61ef4932419'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:34.016015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ac92b862ae4d988409b01c3adfef75', 'request_id': 'req-e4162d58-c295-4ebb-ae35-e61ef4932419'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:34.026438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ac92b862ae4d988409b01c3adfef75', 'request_id': 'req-e4162d58-c295-4ebb-ae35-e61ef4932419'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:34.162855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] The token cache is being invalidate because project b3ac92b862ae4d988409b01c3adfef75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:34.166746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:34.169490 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e4162d58-c295-4ebb-ae35-e61ef4932419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:34.180352 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1080/2176] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:33 2022] DELETE /identity/v3/projects/b3ac92b862ae4d988409b01c3adfef75 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:12:35.364641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:35.367244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:35.410767 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105450-8b44-4880-a71a-74c1341f1617 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:35.491833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:35.517715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:35.518671 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:35.519059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:35.519832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] PATH_INFO: `/v3/users/7773de8f0ee743c9881d441e8e9d8fc2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:35.522777 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7773de8f0ee743c9881d441e8e9d8fc2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:35.525161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:35.854971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7773de8f0ee743c9881d441e8e9d8fc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:35.860345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7773de8f0ee743c9881d441e8e9d8fc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:35.899850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7773de8f0ee743c9881d441e8e9d8fc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:35.901782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7773de8f0ee743c9881d441e8e9d8fc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:35.966267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7773de8f0ee743c9881d441e8e9d8fc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:36.109563 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1097/2177] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:35 2022] DELETE /identity/v3/users/7773de8f0ee743c9881d441e8e9d8fc2 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:36.474498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:36.476014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:36.476757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:36.557999 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:36.563735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:36.735143 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1081/2178] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:36.766488 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:36.768068 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:36.892761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2d57-11dc-4afb-8922-ba9c753332fe admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:36.903215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:36.904521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:36.905458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:36.906101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:36.925648 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:36.929060 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:36.949398 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1098/2179] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:37.179120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:37.181281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:37.203683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd532-faf8-4abe-9d67-15cbddc67bc3 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:37.219106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:37.219106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:37.219106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:37.219106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] PATH_INFO: `/v3/projects/a0dd3a3ba566479d94d5438d890da279` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:37.228191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0dd3a3ba566479d94d5438d890da279)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:37.248044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:37.454876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0dd3a3ba566479d94d5438d890da279', 'request_id': 'req-52243d34-cc4d-436b-a1cb-fbe797162799'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.466597 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0dd3a3ba566479d94d5438d890da279', 'request_id': 'req-52243d34-cc4d-436b-a1cb-fbe797162799'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.475413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0dd3a3ba566479d94d5438d890da279', 'request_id': 'req-52243d34-cc4d-436b-a1cb-fbe797162799'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.478978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:37.568616 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] The token cache is being invalidate because project a0dd3a3ba566479d94d5438d890da279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:37.569037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.570014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-52243d34-cc4d-436b-a1cb-fbe797162799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.571795 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1082/2180] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:37 2022] DELETE /identity/v3/projects/a0dd3a3ba566479d94d5438d890da279 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:37.576711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:37.605487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76be43-8217-4df2-bf9a-96af1d254c8b service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:37.693289 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:37.710207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:37.711400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:37.711924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:37.712421 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] PATH_INFO: `/v3/users/7424867fb1f94222b60e82d6df82a253` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:37.714495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7424867fb1f94222b60e82d6df82a253)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:37.718121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:37.976428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7424867fb1f94222b60e82d6df82a253'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.977676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7424867fb1f94222b60e82d6df82a253'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.993170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7424867fb1f94222b60e82d6df82a253'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:37.994462 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7424867fb1f94222b60e82d6df82a253'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:38.014915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7424867fb1f94222b60e82d6df82a253'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:38.072111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.072427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.072726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.113237 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1099/2181] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:37 2022] DELETE /identity/v3/users/7424867fb1f94222b60e82d6df82a253 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:38.113350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] MFA Rules not processed for user `efbfd82026f14adb91bead1b95154448`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:38.113350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:38.218192 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1083/2182] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:38.222867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:38.224355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:38.421971 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64626b59-3dea-4ae2-9d8e-1599242c3449 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:38.428865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:38.430004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.431330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.431330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.431995 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:38.435912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:38.591889 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1100/2183] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:12:38 2022] POST /identity/v3/projects => generated 383 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:38.595741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:38.614157 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff90e9-2356-40d6-ab56-f9e35ab16adf None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:38.630731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:38.643795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:38.768812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:38.770016 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.770497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.770934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.771668 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:38.774954 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:38.822285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:38.841550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b448a-bb8d-4c22-9174-5e81d721b92e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:38.848587 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:38.849258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.849607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.850054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] PATH_INFO: `/v3/projects/93aecb6a030847cd868359b1201b5121` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.852003 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93aecb6a030847cd868359b1201b5121)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:38.856740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:38.889939 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1084/2184] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:12:38 2022] POST /identity/v3/users => generated 298 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:38.895870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:38.909568 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b678600-f87d-4dc1-bf8f-78ea600cfa79 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:38.916202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:38.919885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.919885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.919885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.919885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:38.920564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:38.940363 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1085/2185] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:38.942966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:38.956236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4cdf-c32e-40bf-810c-f55dfb738d84 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:38.963038 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:38.963808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:38.964096 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:38.964371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] PATH_INFO: `/v3/projects/860ce309152043c7b57fc66f1b3bb858/users/a6345ea169d248a89795ae42cd3f7df9/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:38.987206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860ce309152043c7b57fc66f1b3bb858, user_id=a6345ea169d248a89795ae42cd3f7df9, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.003177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.006346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93aecb6a030847cd868359b1201b5121', 'request_id': 'req-36a04e6d-fc13-435e-94dd-a5c463958436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:39.019352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93aecb6a030847cd868359b1201b5121', 'request_id': 'req-36a04e6d-fc13-435e-94dd-a5c463958436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:39.144788 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1086/2186] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:38 2022] PUT /identity/v3/projects/860ce309152043c7b57fc66f1b3bb858/users/a6345ea169d248a89795ae42cd3f7df9/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:39.151215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.157031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93aecb6a030847cd868359b1201b5121', 'request_id': 'req-36a04e6d-fc13-435e-94dd-a5c463958436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:39.193545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] The token cache is being invalidate because project 93aecb6a030847cd868359b1201b5121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:39.194358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:39.195904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-36a04e6d-fc13-435e-94dd-a5c463958436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:39.199445 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1101/2187] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:38 2022] DELETE /identity/v3/projects/93aecb6a030847cd868359b1201b5121 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:39.205561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f077-d05f-4c53-9095-e29fbf6990c1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.206149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f077-d05f-4c53-9095-e29fbf6990c1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.206694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f077-d05f-4c53-9095-e29fbf6990c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.241266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4955f077-d05f-4c53-9095-e29fbf6990c1 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:39.243368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4955f077-d05f-4c53-9095-e29fbf6990c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:39.245483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:39.299863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421f95c-34b7-44cd-92d4-a9e9c60c6a81 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.307563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.308571 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.309022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.309457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.310570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.315325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.337862 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1087/2188] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:12:39.339452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.343320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:39.366206 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f2c8-491d-4207-9bac-c3f03fe0b43f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.368503 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1102/2189] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:39.373124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.373897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.374188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.374486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] PATH_INFO: `/v3/projects/860ce309152043c7b57fc66f1b3bb858/users/a6345ea169d248a89795ae42cd3f7df9/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.379122 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860ce309152043c7b57fc66f1b3bb858, user_id=a6345ea169d248a89795ae42cd3f7df9, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.381519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.389012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.389468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.389768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.396828 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1088/2190] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:39 2022] PUT /identity/v3/projects/860ce309152043c7b57fc66f1b3bb858/users/a6345ea169d248a89795ae42cd3f7df9/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:39.423737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:39.426109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:39.514936 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1103/2191] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:39.519477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.531101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800cbf-b7fe-4762-af02-7bd0daca10d0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.537654 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.538433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.538808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.539246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.540065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:12:39.540493 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.542502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.557061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1089/2192] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:12:39.561886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.575365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de2eb9-a560-41db-bb10-0076ef5a3768 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.583672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.584717 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.585229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.585715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.587016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.590912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.744691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:39.754726 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1104/2193] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:12:39 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:39.757415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.770026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491ebb-ae57-415f-bede-6c7f55e808cc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.837172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.837938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.838286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.838653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.839518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.842254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:39.972336 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1090/2194] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:12:39 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:39.973813 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:39.986651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f6531-4d42-4868-892c-68aa44fdcdc1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:39.992243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:39.993005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:39.993285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:39.998040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:39.998040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:39.998040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:40.008604 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1105/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:40.013197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:40.024364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b25-a559-4328-b640-45b43917c230 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:40.029138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:40.029761 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:40.030041 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:40.030316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:40.032322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:40.032680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bcd24-fa87-41a7-95b0-c9b47b8a7df5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:40.046322 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1091/2196] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:12:40.051428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:40.062371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a62be2-bb93-4b27-a388-25cc5a6916ba admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:40.068666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:40.069343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:40.069810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:40.070095 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] PATH_INFO: `/v3/projects/a0af9615d9ba419aab60cdec7f7c4c71/users/4da19993aab04c758c7d8e7659dad093/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:40.092426 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0af9615d9ba419aab60cdec7f7c4c71, user_id=4da19993aab04c758c7d8e7659dad093, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:40.098102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:40.292488 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1106/2197] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:40 2022] PUT /identity/v3/projects/a0af9615d9ba419aab60cdec7f7c4c71/users/4da19993aab04c758c7d8e7659dad093/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:40.312899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:40.313186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:40.313468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:40.355766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] MFA Rules not processed for user `4da19993aab04c758c7d8e7659dad093`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:40.357068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:40.462846 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1092/2198] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:40.492820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:40.496821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:40.657993 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071f35c-00f6-42d7-8d78-aad3081a46fc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:40.667144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:40.667144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:40.667565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:40.668104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:40.701728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:40.709718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651d11-008f-4c0e-b2cf-4594911acaa8 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:40.725587 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1107/2199] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:40.751740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:40.777117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:40.863630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af544-01a4-43c6-9b26-5aef69741478 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:40.872941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:40.874708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:40.874934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:40.875379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] PATH_INFO: `/v3/users/4da19993aab04c758c7d8e7659dad093` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:40.878196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da19993aab04c758c7d8e7659dad093)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:40.880811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:41.196925 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da19993aab04c758c7d8e7659dad093'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:41.198364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da19993aab04c758c7d8e7659dad093'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:41.222455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da19993aab04c758c7d8e7659dad093'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:41.355313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da19993aab04c758c7d8e7659dad093'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:41.372128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da19993aab04c758c7d8e7659dad093'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:41.388678 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1093/2200] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:40 2022] DELETE /identity/v3/users/4da19993aab04c758c7d8e7659dad093 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:12:41.394012 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb75e5-5f3a-4205-a7ef-57f68dd77a40 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:41.394557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb75e5-5f3a-4205-a7ef-57f68dd77a40 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:41.394557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb75e5-5f3a-4205-a7ef-57f68dd77a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:41.424603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7edb75e5-5f3a-4205-a7ef-57f68dd77a40 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:41.426811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7edb75e5-5f3a-4205-a7ef-57f68dd77a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:41.519447 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1108/2201] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:41.532725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:41.535357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:41.699393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c431-f9c2-46a0-9419-629196134497 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:41.703742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:41.704958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:41.705930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:41.706217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:41.721619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:41.724020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:41.769995 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1094/2202] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:41.970955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:41.971132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:41.971946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:41.996305 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:41.997604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:42.007322 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1109/2203] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:42.035031 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:42.037391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:42.113891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:42.173783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314ba31-c93d-441b-9037-4d6b64eb539a service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:42.210405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:42.211061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:42.211350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:42.211655 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:42.238800 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:42.241218 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d6d32-d333-4037-99f9-85e39cb411a1 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:42.276501 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1095/2204] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:42.566507 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:42.569924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:42.644363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7239f7-ac8b-4986-b38e-809f7c178f18 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:42.654057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:42.657969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:42.657969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:42.657969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] PATH_INFO: `/v3/projects/a0af9615d9ba419aab60cdec7f7c4c71` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:42.671405 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0af9615d9ba419aab60cdec7f7c4c71)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:42.679547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:42.887353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0af9615d9ba419aab60cdec7f7c4c71', 'request_id': 'req-736b9f0f-78cf-4ab5-a52a-7297e4905716'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:42.900084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:42.903841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0af9615d9ba419aab60cdec7f7c4c71', 'request_id': 'req-736b9f0f-78cf-4ab5-a52a-7297e4905716'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:43.042317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0af9615d9ba419aab60cdec7f7c4c71', 'request_id': 'req-736b9f0f-78cf-4ab5-a52a-7297e4905716'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:43.059903 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] The token cache is being invalidate because project a0af9615d9ba419aab60cdec7f7c4c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:43.060240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:43.060921 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-736b9f0f-78cf-4ab5-a52a-7297e4905716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:43.062558 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1110/2205] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:42 2022] DELETE /identity/v3/projects/a0af9615d9ba419aab60cdec7f7c4c71 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:12:43.071466 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.073214 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.074240 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.113530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:43.115099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:43.203508 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1096/2206] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:43.213821 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.214111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.214391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.239379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:43.240675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:43.255639 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1111/2207] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:43.260798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.273844 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2923f3-5246-49c6-b895-5d3efd61166a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.278683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.279414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.279804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.280166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.280912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:12:43.281292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.283109 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:43.296773 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1097/2208] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:12:43.302279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.321554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5809b-4fb7-4ba4-9b43-e1e4e7d4e466 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.330844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.331999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.332494 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.332983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.334169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.337858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:43.472601 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1112/2209] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:12:43 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:12:43.478182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.509843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabdfc8-92af-4003-843a-935ffc071576 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.634762 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.636593 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.636593 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.637146 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.638415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.642659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:43.787760 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1098/2210] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:12:43 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:12:43.792876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.816692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270a42-490a-4110-9ea8-68c0feadbe65 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.823302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.824314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.824850 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.825343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.826228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.828734 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:43.854100 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1113/2211] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:12:43.855050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.870103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5153-d172-4821-a63b-dd3c27b939d2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.877729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.879236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.880018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.880582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.881784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.885186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4ff3f-3651-444c-beda-5e6981e55132 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:43.906528 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1099/2212] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:12:43.913028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:43.926375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759f9b8-377d-4168-9421-81e165c40195 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:43.934768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:43.935862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:43.936337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:43.938015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] PATH_INFO: `/v3/projects/c29fca287584407b841cf77a1af54194/users/35a04ae0524d437d9dd0ec78efd7687a/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:43.963942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29fca287584407b841cf77a1af54194, user_id=35a04ae0524d437d9dd0ec78efd7687a, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:43.968015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:44.066033 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1114/2213] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:12:43 2022] PUT /identity/v3/projects/c29fca287584407b841cf77a1af54194/users/35a04ae0524d437d9dd0ec78efd7687a/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:12:44.086534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e43b-a69f-4a5e-acb3-27821ce02eb2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:44.088196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e43b-a69f-4a5e-acb3-27821ce02eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:44.089110 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e43b-a69f-4a5e-acb3-27821ce02eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:44.144201 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4318e43b-a69f-4a5e-acb3-27821ce02eb2 None None] MFA Rules not processed for user `35a04ae0524d437d9dd0ec78efd7687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:44.147501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4318e43b-a69f-4a5e-acb3-27821ce02eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:44.176994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:44.351967 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1100/2214] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:44.369025 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:44.374797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:44.543547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06756f6e-782d-4f4b-92a1-72d705a03a87 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:44.551497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:44.552843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:44.553370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:44.554205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:44.573975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:44.576943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163a94f-c13c-4061-b0d5-c7ae4a234e85 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:44.597774 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1115/2215] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:44.623032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd2d2-eb06-4d46-a5a2-e799fc3eb1cb None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:44.623365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd2d2-eb06-4d46-a5a2-e799fc3eb1cb None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:44.623647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd2d2-eb06-4d46-a5a2-e799fc3eb1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:44.683406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-3dbdd2d2-eb06-4d46-a5a2-e799fc3eb1cb None None] MFA Rules not processed for user `a4a4230574c846a9b4f7edd240317ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:44.684432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3dbdd2d2-eb06-4d46-a5a2-e799fc3eb1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:45.078399 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1101/2216] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:47.401579 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:47.402328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:47.402328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:47.459301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:47.462418 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:47.560901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:47.561440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:47.561440 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:47.623877 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1116/2217] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:47.636773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] MFA Rules not processed for user `a6345ea169d248a89795ae42cd3f7df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:47.636773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:47.663827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:47.685762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:47.753056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:47.818746 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1102/2218] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:47.841940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:47.846398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be02a2-93fd-4211-8898-7a7ea22a2f08 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:47.847126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:47.853712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:47.854460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:47.854849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:47.855133 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:47.869256 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:47.872318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7d2ce-7340-4946-9d2d-0a6e9f016eaf service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:47.889026 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1117/2219] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:47.946774 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:47.958942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bc7fe-2123-44ce-ae12-cec9d78d55e7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:47.966933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:47.968449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:47.969252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:47.970410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:47.974123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:48.076736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:48.078449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:48.090078 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1103/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:48.494335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:48.495340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:48.495847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:48.523659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:48.524951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:48.624352 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1118/2221] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:12:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:12:48.641394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:48.643522 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:48.784335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1e3e-bce0-483f-b96a-6e3199dff2f3 service glance] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:48.793470 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:48.794481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:48.794865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:48.795148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:48.809689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:48.809689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab8868-e004-413d-bde8-546a7599526d service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:48.834486 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1104/2222] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:12:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 30 01:12:50.485523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:50.519319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41940f-80a3-4733-8ade-66ecc4309541 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:50.541156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:50.542297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:50.542805 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:50.543307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] PATH_INFO: `/v3/users/35a04ae0524d437d9dd0ec78efd7687a` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:50.547978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a04ae0524d437d9dd0ec78efd7687a)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:50.552326 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:50.910828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a04ae0524d437d9dd0ec78efd7687a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:50.917521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a04ae0524d437d9dd0ec78efd7687a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:50.941192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a04ae0524d437d9dd0ec78efd7687a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:50.943106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a04ae0524d437d9dd0ec78efd7687a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:50.987000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a04ae0524d437d9dd0ec78efd7687a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:51.128431 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1119/2223] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:12:50 2022] DELETE /identity/v3/users/35a04ae0524d437d9dd0ec78efd7687a => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:12:51.135700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:51.137159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:51.138174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:51.191213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:12:51.194293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:51.362762 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1105/2224] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:12:51.382098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:51.384676 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:51.467835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:51.538048 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170a36-96d3-4cde-9371-ae56c4b4274c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:51.543479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:51.544646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:51.545175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:51.545771 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:51.560248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:51.562252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:51.581049 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1120/2225] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:52.577140 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:52.578130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:52.657114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ababb-8a36-4808-8d89-19f4cc14ce08 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:52.697918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:52.699902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:52.701320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:52.702098 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] PATH_INFO: `/v3/projects/c29fca287584407b841cf77a1af54194` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:52.704602 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29fca287584407b841cf77a1af54194)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:52.711538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:52.895908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:52.929474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29fca287584407b841cf77a1af54194', 'request_id': 'req-227c43da-462c-41a6-8d66-a3a510b2918b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:52.937596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29fca287584407b841cf77a1af54194', 'request_id': 'req-227c43da-462c-41a6-8d66-a3a510b2918b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:52.946409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29fca287584407b841cf77a1af54194', 'request_id': 'req-227c43da-462c-41a6-8d66-a3a510b2918b'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:53.031013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] The token cache is being invalidate because project c29fca287584407b841cf77a1af54194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:12:53.031309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:53.032879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-227c43da-462c-41a6-8d66-a3a510b2918b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:12:53.037297 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1106/2226] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:12:52 2022] DELETE /identity/v3/projects/c29fca287584407b841cf77a1af54194 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:13:01.942491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=14 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:02.409452 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:02.410990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:02.485585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0fe08-6824-4d0a-8835-4c84732ffcdb service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:02.564839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:02.565478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:02.565765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:02.566045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:02.568015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:02.581264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:02.587030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd25953-03ff-48f8-90c0-a8669428ffd4 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:02.594319 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1121/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:13:03.171557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:04.188963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:04.189960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:04.189960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:04.213937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] MFA Rules not processed for user `efbfd82026f14adb91bead1b95154448`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:04.217928 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:04.316073 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1107/2228] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:13:08.772351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e0dc-785d-46f2-81c1-2bd7379fb732 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:08.773161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e0dc-785d-46f2-81c1-2bd7379fb732 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:08.773355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e0dc-785d-46f2-81c1-2bd7379fb732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:08.807992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e127e0dc-785d-46f2-81c1-2bd7379fb732 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:08.810391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e127e0dc-785d-46f2-81c1-2bd7379fb732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:08.920820 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1122/2229] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:13:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:13:08.935806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:08.938565 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:09.023674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459248fe-d758-49c3-8803-784c56ddbbe4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:09.029773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:09.030430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:09.036367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:09.036367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:09.041279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:09.043232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a586507-064d-4631-921f-7b39a9eaf64c service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:09.057115 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1108/2230] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:13:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:13:09.207867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:09.207867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:09.207867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:09.214949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] MFA Rules not processed for user `9254fc77e32147fdb3a28422b8458039`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:09.216324 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:09.283331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:09.369325 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1123/2231] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:13:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:13:19.049808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=57 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:19.362901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:31.088058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe2b1-d6d4-41d1-8553-43ee471a8dfe None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:31.088995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe2b1-d6d4-41d1-8553-43ee471a8dfe None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:31.088995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe2b1-d6d4-41d1-8553-43ee471a8dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:31.112778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1eabe2b1-d6d4-41d1-8553-43ee471a8dfe None None] MFA Rules not processed for user `2cfba439a18a4c8e8fa0021380128447`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:31.114342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1eabe2b1-d6d4-41d1-8553-43ee471a8dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:31.206482 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1109/2232] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:13:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:13:38.586467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:38.588057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:38.674811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db774d94-958e-4c17-9b4d-1e4ac5331dfd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:38.679640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:38.684213 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:38.684637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:38.686938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:38.688721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:38.778905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:38.780878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:38.795160 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1124/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:13:41.194804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=25 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:47.749693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63b68-a796-4895-81df-835c16a63202 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:47.750861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63b68-a796-4895-81df-835c16a63202 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:47.750861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63b68-a796-4895-81df-835c16a63202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:47.803502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-76d63b68-a796-4895-81df-835c16a63202 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:47.803502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-76d63b68-a796-4895-81df-835c16a63202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:47.860664 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1110/2234] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:13:47.882350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:47.883994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:47.978074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9c347-fe55-4342-9e54-6a8f786b63b9 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:47.986635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:47.987343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:47.987765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:47.988198 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:47.998851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:48.000616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:48.012755 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1125/2235] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:13:53.243514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20200-3eda-4cb9-827f-81e5e38ec37e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:53.244227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20200-3eda-4cb9-827f-81e5e38ec37e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:53.244227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20200-3eda-4cb9-827f-81e5e38ec37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:53.266329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-17f20200-3eda-4cb9-827f-81e5e38ec37e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:53.267475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-17f20200-3eda-4cb9-827f-81e5e38ec37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:53.276034 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1111/2236] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:13:53.302397 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:53.304948 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:53.329034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22927800-84ea-4296-9256-00dc875d9ef8 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:53.335667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:53.336335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:53.336712 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:53.336999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:53.347784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:53.349463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:53.360764 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1126/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:13:57.097792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd38dbd-e032-4d64-a14a-8b3b8de51bdb None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:57.098482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd38dbd-e032-4d64-a14a-8b3b8de51bdb None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:57.098482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd38dbd-e032-4d64-a14a-8b3b8de51bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:57.120316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5fd38dbd-e032-4d64-a14a-8b3b8de51bdb None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:13:57.121558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5fd38dbd-e032-4d64-a14a-8b3b8de51bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:57.131006 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1112/2238] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:13:57.160101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:57.178223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c44e-7b94-4532-b673-fc814573f118 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:57.185569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:57.186280 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:57.186565 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:57.186843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:57.196468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:57.198317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:57.209853 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1127/2239] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:00.418903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:00.420306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:00.420306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:00.452114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:00.454838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:00.487388 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1113/2240] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:14:00.523440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:00.535502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:00.550489 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb2c8-7ef9-481f-ab72-f900060a7ce4 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:00.557990 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:00.559382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:00.560297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:00.560845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:00.576250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:00.579375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:00.603034 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1128/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:02.688945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:02.691729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:02.704452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd1b8b-63e6-4464-bec0-c8bc4927dc30 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:02.709686 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:02.710323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:02.710610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:02.710889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] PATH_INFO: `/v3/users/a4a4230574c846a9b4f7edd240317ee9` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:02.712154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a4230574c846a9b4f7edd240317ee9)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:02.714219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:02.951776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a4230574c846a9b4f7edd240317ee9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:02.952391 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a4230574c846a9b4f7edd240317ee9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:02.968658 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a4230574c846a9b4f7edd240317ee9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.059664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a4230574c846a9b4f7edd240317ee9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.068823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-f7033927-37f2-47b2-a756-8be61e68666e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a4230574c846a9b4f7edd240317ee9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.077989 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1114/2242] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:14:02 2022] DELETE /identity/v3/users/a4a4230574c846a9b4f7edd240317ee9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:14:03.487005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:03.499340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326afe54-926b-4fa2-8e7b-373f91c9aa86 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:03.580138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:03.580802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:03.581090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:03.581365 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] PATH_INFO: `/v3/projects/95c63194a18f4768abe23f55608ec8c6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:03.582711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c63194a18f4768abe23f55608ec8c6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:03.584741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:03.718576 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:03.719177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:03.719177 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:03.743318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:03.744913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:03.754640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c63194a18f4768abe23f55608ec8c6', 'request_id': 'req-aad0b6a1-1d0e-48df-ba5d-10897af67991'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.762227 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c63194a18f4768abe23f55608ec8c6', 'request_id': 'req-aad0b6a1-1d0e-48df-ba5d-10897af67991'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.829138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:03.843763 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1115/2243] 10.222.0.61 () {58 vars in 1016 bytes} [Sun Oct 30 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:14:03.882839 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c63194a18f4768abe23f55608ec8c6', 'request_id': 'req-aad0b6a1-1d0e-48df-ba5d-10897af67991'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.884525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:03.885987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:03.906098 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] The token cache is being invalidate because project 95c63194a18f4768abe23f55608ec8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:14:03.906626 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.907810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aad0b6a1-1d0e-48df-ba5d-10897af67991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:03.911095 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1129/2244] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:14:03 2022] DELETE /identity/v3/projects/95c63194a18f4768abe23f55608ec8c6 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:14:03.917270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1549-c0ec-4e34-8889-dd46603e9e51 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:03.917700 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1549-c0ec-4e34-8889-dd46603e9e51 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:03.918158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1549-c0ec-4e34-8889-dd46603e9e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:03.951262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f24a1549-c0ec-4e34-8889-dd46603e9e51 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:03.953404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f24a1549-c0ec-4e34-8889-dd46603e9e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:03.965421 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1130/2245] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:14:03.973873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:03.974166 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:03.974441 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:03.999299 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:04.001008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:04.008642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b329-ba0a-4a52-bcd8-2dcdc28f46b0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.013310 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1131/2246] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:14:04.017125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.018352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.018773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.019223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.020149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.022269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:04.034696 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e25fce-6663-4ce4-857c-8c97a7267e19 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.039365 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.041109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.041520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.041818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.042210 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.042607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.043343 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:14:04.043797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.046347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.054356 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1116/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:14:04.065284 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1132/2248] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:14:04.071877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.086497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cea57-c25e-47c0-a498-23ab29cd06cc service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.098506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.099769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.100315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.100823 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.102121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.106349 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.222980 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1117/2249] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:14:04 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:14:04.227662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.245278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe88a2-380a-45b3-bea4-97f045c86467 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.321670 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:04.333017 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.346659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.346659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.346659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.346659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.346659 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.459392 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1133/2250] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:14:04 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:14:04.464085 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.475689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654ee83-4597-4820-a1c5-c23f0ce6cad4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.480777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.481406 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.481679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.481951 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.482631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.484346 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.496583 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1118/2251] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:14:04.500992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.512601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7922cc5-cac4-44b9-b4d1-f2d69254bf18 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.518882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.519570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.519945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.526015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.526015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.526015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2e6b-fc8a-4b49-b406-bae5efd66ca2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.536133 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1134/2252] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:14:04.540647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.552960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cf2-eee5-48b8-a798-a791000b7576 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.557946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.558748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.559123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.559500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] PATH_INFO: `/v3/projects/6e74d6179e574138841007195fd72400/users/1d21128dee354a04bfb008efe6b60899/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.578074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e74d6179e574138841007195fd72400, user_id=1d21128dee354a04bfb008efe6b60899, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.581260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.649172 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1119/2253] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:14:04 2022] PUT /identity/v3/projects/6e74d6179e574138841007195fd72400/users/1d21128dee354a04bfb008efe6b60899/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:14:04.655533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.655887 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.656191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.677090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:04.678424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:04.760315 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1135/2254] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:14:04.773366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:04.775289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:04.847376 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5cf48-1872-4596-8bd3-83b9fabf3142 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:04.851866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:04.852499 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:04.852776 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:04.853046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:04.863383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:04.864953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:04.877933 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1120/2255] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:09.166362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:09.191400 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb983f-66eb-4b48-8cbc-20ca99aa1d43 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:09.196804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:09.197733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:09.198192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:09.198623 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:09.204341 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:09.337237 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:09.340406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4f0b6-2cba-4bdb-8595-f868058db809 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:09.364606 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1136/2256] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:10.679683 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:10.701167 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d4928-59e5-4caa-a7f2-be6e20ebee9b service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:10.708425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:10.709599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:10.710175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:10.710756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:10.714222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:10.744420 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:10.747575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:10.769291 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1121/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:14:12.774835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60109323-4364-49f6-b947-2773a77e990c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:12.775702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60109323-4364-49f6-b947-2773a77e990c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:12.776063 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60109323-4364-49f6-b947-2773a77e990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:12.814496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-60109323-4364-49f6-b947-2773a77e990c None None] MFA Rules not processed for user `1d21128dee354a04bfb008efe6b60899`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:12.816121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-60109323-4364-49f6-b947-2773a77e990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:12.916642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:12.942365 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1137/2258] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:14:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:14:12.961684 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:12.963176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:12.978327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc75e8b-05a9-4e60-b81a-ef5cffded928 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:12.985039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:12.985557 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:12.985838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:12.986114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:12.996702 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:13.006824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:13.006824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:13.010832 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1122/2259] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:17.218803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:17.219501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:17.219501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:17.247624 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] MFA Rules not processed for user `1d21128dee354a04bfb008efe6b60899`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:14:17.249743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:17.268944 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1138/2260] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:14:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:14:23.224465 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:27.085371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:27.088389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:27.119504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df14ec0-677d-407d-af52-cb12e4e36aea service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:27.134068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:27.136617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:27.136617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:27.136617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:27.140159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:27.226572 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:27.228358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:27.239295 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=9 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:27.244191 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1123/2261] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:14:36.382586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:36.399427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca761b23-f63e-456b-af76-5be5d969ae05 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:36.405467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:36.406205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:36.406545 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:36.406899 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:36.409102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:36.422145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:36.423308 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:36.440936 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1139/2262] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:37.237329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=28 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:43.880752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:43.882838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:43.983275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc01db-8d83-44b4-8252-ed589ac6b221 service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:43.988990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:43.989976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:43.990337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:43.990689 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:43.992994 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:44.012554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:44.015243 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:44.033273 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1124/2263] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:44.045953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:44.048715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:44.078628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bdc0c-2b74-4d51-a624-dc2c1b8b0700 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:44.085149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:44.085944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:44.086325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:44.086695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] PATH_INFO: `/v3/users/9254fc77e32147fdb3a28422b8458039` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:44.088362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9254fc77e32147fdb3a28422b8458039)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:44.090809 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:44.362778 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9254fc77e32147fdb3a28422b8458039'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:44.363501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9254fc77e32147fdb3a28422b8458039'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:44.373069 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9254fc77e32147fdb3a28422b8458039'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:44.374029 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9254fc77e32147fdb3a28422b8458039'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:44.392035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9254fc77e32147fdb3a28422b8458039'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:44.523531 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1140/2264] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:14:44 2022] DELETE /identity/v3/users/9254fc77e32147fdb3a28422b8458039 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:14:44.853431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:44.872045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac93d-118f-41c0-ae6d-ec6d95fa6456 service placement] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:44.952960 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:44.953801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:44.953915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:44.954162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] PATH_INFO: `/v3/projects/ec0e8d209ae34edca0d7dc5dae71129a` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:44.955360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0e8d209ae34edca0d7dc5dae71129a)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:44.957359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:45.084222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0e8d209ae34edca0d7dc5dae71129a', 'request_id': 'req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:45.098044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0e8d209ae34edca0d7dc5dae71129a', 'request_id': 'req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:45.106946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0e8d209ae34edca0d7dc5dae71129a', 'request_id': 'req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:45.173040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] The token cache is being invalidate because project ec0e8d209ae34edca0d7dc5dae71129a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:14:45.173392 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:45.174149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:14:45.175983 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1125/2265] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:14:44 2022] DELETE /identity/v3/projects/ec0e8d209ae34edca0d7dc5dae71129a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:14:48.384309 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:48.387001 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:48.519715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef300961-a741-4328-926e-18ff79b85aaa admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:48.598049 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:48.620274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:48.621043 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:48.621398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:48.621788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:48.624205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:48.639468 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:48.641316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0ded7-f345-4f46-b309-49e959cfdaa6 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:48.658441 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1141/2266] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:55.168792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:14:56.352126 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:14:56.353731 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:56.421106 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea923-35af-4d8a-99e5-c975e7fe4f88 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:14:56.425515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:14:56.426190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:14:56.426678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:14:56.427008 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:14:56.429134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:14:56.503544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:14:56.504977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4cb9d-92b4-4d7d-992b-c52b8b2009db service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:14:56.516424 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1126/2267] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:14:58.840181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:00.936363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930ba2-ea22-48bd-aa3d-39b68330ad92 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:00.937096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930ba2-ea22-48bd-aa3d-39b68330ad92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:00.937096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930ba2-ea22-48bd-aa3d-39b68330ad92 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:00.939148 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1142/2268] 10.222.0.26 () {54 vars in 961 bytes} [Sun Oct 30 01:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:15:00.946171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:00.946801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:00.946801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:00.968778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] MFA Rules not processed for user `0133918a941b4c3ba07842d5b4deabbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:00.970118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:01.045545 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1127/2269] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Oct 30 01:15:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:01.053495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f930ba2-ea22-48bd-aa3d-39b68330ad92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:01.069340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f930ba2-ea22-48bd-aa3d-39b68330ad92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:01.075710 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:01.076345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:01.076633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:01.076917 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:01.079072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:01.092689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:01.094623 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:01.096751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:01.099399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:01.106797 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1143/2270] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:15:01.118612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463c31c-f6db-47fc-8999-a8a0f34045d9 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:01.126228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:01.127316 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:01.127852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:01.128341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] PATH_INFO: `/v3/users/2cfba439a18a4c8e8fa0021380128447` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:01.130695 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfba439a18a4c8e8fa0021380128447)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:01.132673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:01.381550 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:01.388190 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfba439a18a4c8e8fa0021380128447'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:01.389015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfba439a18a4c8e8fa0021380128447'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:01.404742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfba439a18a4c8e8fa0021380128447'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:01.540895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfba439a18a4c8e8fa0021380128447'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:01.553283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-53c36a88-6a32-4800-beb0-43184fb11cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfba439a18a4c8e8fa0021380128447'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:01.566705 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1128/2271] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:15:01 2022] DELETE /identity/v3/users/2cfba439a18a4c8e8fa0021380128447 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:01.910697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:01.929229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35194b-8a96-4cdd-bcf8-4ad444b2789c service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:02.011911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:02.012695 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.012862 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.013158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] PATH_INFO: `/v3/projects/82fc550e1caf445d8027e8051e0b4d30` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.014537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] RBAC: Authorizing `identity:delete_project(project_id=82fc550e1caf445d8027e8051e0b4d30)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:02.016617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:02.165467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82fc550e1caf445d8027e8051e0b4d30', 'request_id': 'req-536bb57b-4c7f-4bb2-9918-4183b7abdeba'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:02.173555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82fc550e1caf445d8027e8051e0b4d30', 'request_id': 'req-536bb57b-4c7f-4bb2-9918-4183b7abdeba'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:02.256560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82fc550e1caf445d8027e8051e0b4d30', 'request_id': 'req-536bb57b-4c7f-4bb2-9918-4183b7abdeba'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:02.271530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] The token cache is being invalidate because project 82fc550e1caf445d8027e8051e0b4d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:15:02.271875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:02.272585 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-536bb57b-4c7f-4bb2-9918-4183b7abdeba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:02.274180 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1144/2272] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:15:01 2022] DELETE /identity/v3/projects/82fc550e1caf445d8027e8051e0b4d30 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:02.280864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.281384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.281619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.309775 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:02.311813 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:02.402498 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1129/2273] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:15:02.411358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.411891 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.412265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.435840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:02.437022 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:02.445669 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1145/2274] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:02.451175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:02.464326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc40919-bc53-4dd4-8e08-e769085e6ade None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:02.469581 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:02.470219 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.470501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.470780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.471469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:15:02.471882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:02.473588 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:02.487375 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1130/2275] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:15:02.494244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:02.510951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b6d38-1ce7-4dd3-991f-97bcb0220bbd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:02.519066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:02.520184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.520693 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.521194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.522389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:02.526787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:02.700926 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1146/2276] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:15:02 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:15:02.707083 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:02.719728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e684d33-9485-43c0-b43f-1c3b6de4537e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:02.825533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:02.826586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:02.827068 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:02.827561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:02.828846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:02.832652 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:02.996331 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1131/2277] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:15:02 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:15:03.000932 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:03.012438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618ac-67ee-41e0-aa37-b67e1ca3df6e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:03.017944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:03.018636 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:03.018954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:03.019263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:03.020034 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:03.022260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:03.034006 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1147/2278] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:15:03.038473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:03.050142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e9066-0b87-4a86-9e1c-91a62e4a8c4e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:03.055444 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:03.056113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:03.056398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:03.056677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:03.057977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:03.059657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c3f12-fca8-4688-ab8e-1e6fb096d8bf admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:03.072156 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1132/2279] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:15:03.076976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:03.088192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827a916-03f9-4acd-846e-39b95024402a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:03.093554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:03.094248 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:03.094535 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:03.094816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] PATH_INFO: `/v3/projects/cd37394129e346a8804d667856309041/users/d09696979ec94a2699d7cb1e2aa77a64/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:03.114157 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd37394129e346a8804d667856309041, user_id=d09696979ec94a2699d7cb1e2aa77a64, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:03.117547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:03.229185 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1148/2280] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:15:03 2022] PUT /identity/v3/projects/cd37394129e346a8804d667856309041/users/d09696979ec94a2699d7cb1e2aa77a64/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:15:03.234725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ea88-4607-40b6-8f25-6153a9cfc343 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:03.235048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ea88-4607-40b6-8f25-6153a9cfc343 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:03.245178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ea88-4607-40b6-8f25-6153a9cfc343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:03.256870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-91f9ea88-4607-40b6-8f25-6153a9cfc343 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:03.258048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-91f9ea88-4607-40b6-8f25-6153a9cfc343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:03.281816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.328900 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1133/2281] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:03.339388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:03.345054 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:03.358187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.408358 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e7c-742c-4dda-b7b9-60e78960c626 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:03.412637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:03.413281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:03.413568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:03.413851 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:03.424818 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:03.426547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:03.438815 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1149/2282] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:15:10.393910 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daaaf8-1c2c-4a16-8820-3e86cd7c4da7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:10.394511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daaaf8-1c2c-4a16-8820-3e86cd7c4da7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:10.394857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daaaf8-1c2c-4a16-8820-3e86cd7c4da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:10.450278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-57daaaf8-1c2c-4a16-8820-3e86cd7c4da7 None None] MFA Rules not processed for user `d09696979ec94a2699d7cb1e2aa77a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:10.452781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-57daaaf8-1c2c-4a16-8820-3e86cd7c4da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:10.586629 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1134/2283] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:10.621062 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:10.622438 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:10.804216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849594-7144-48f9-9150-23e821820a9d service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:10.813943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:10.816169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:10.816795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:10.817219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:10.828798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:10.830501 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:10.842511 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1150/2284] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:15:15.173533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:15.174063 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:15.175180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:15.201291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] MFA Rules not processed for user `d09696979ec94a2699d7cb1e2aa77a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:15.202592 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:15.214855 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1135/2285] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:15.918364 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:15.919888 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:15.933375 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d961-1d94-4025-941a-577b41722a94 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:15.937456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:15.938081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:15.938374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:15.938679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:15.940530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:15.953126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:15.954625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:15.964897 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1151/2286] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:15:16.435318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:16.437772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:16.465894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:16.468035 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:16.497419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652dacf-6403-4e15-80b5-82def6ac3016 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:16.521587 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:16.522876 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:16.523377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:16.524129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:16.529205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:16.561725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:16.570649 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1e68d-69c9-4f69-b17f-f3d8d8f198e2 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:16.618138 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1136/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:15:16.644128 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b854d-0114-4786-98cc-9fb79531e23f service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:16.656123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:16.657586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:16.658141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:16.659092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:16.686169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:16.710533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:16.714132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:16.736712 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1152/2288] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:15:25.777309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:25.778262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:25.778262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:25.805610 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:25.807753 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:25.879004 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1137/2289] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:15:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:25.939710 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:25.941416 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:25.966540 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:26.033281 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b069b0-f30f-450e-88e9-53484d736df3 service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:26.041078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:26.041728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:26.042026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:26.042318 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:26.053420 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:26.055242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:26.065508 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1153/2290] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:15:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:29.327543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:29.347145 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978990e-d682-415e-a7df-55def8fc35cc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:29.353503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:29.354295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:29.354764 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:29.355230 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:29.357342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:29.372351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:29.374770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:29.430262 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1138/2291] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:15:36.061339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=24 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:38.693059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:38.695885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:38.722471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a7232-6002-4d03-a405-f0672ba0f77a service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:38.728552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:38.729240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:38.729520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:38.729799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] PATH_INFO: `/v3/users/efbfd82026f14adb91bead1b95154448` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:38.731294 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efbfd82026f14adb91bead1b95154448)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:38.733347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:38.938904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efbfd82026f14adb91bead1b95154448'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:38.939514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efbfd82026f14adb91bead1b95154448'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:38.947773 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efbfd82026f14adb91bead1b95154448'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:38.948517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efbfd82026f14adb91bead1b95154448'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:38.962725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efbfd82026f14adb91bead1b95154448'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.056227 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1154/2292] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:15:38 2022] DELETE /identity/v3/users/efbfd82026f14adb91bead1b95154448 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:39.390812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:39.401191 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:39.411716 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd61f-add6-4396-99db-31173ecdd5d9 service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:39.522238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:39.528129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:39.528129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:39.528129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] PATH_INFO: `/v3/projects/77968fc2b85349b2a625ea9bf25cd026` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:39.528129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77968fc2b85349b2a625ea9bf25cd026)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:39.530995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:39.694612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77968fc2b85349b2a625ea9bf25cd026', 'request_id': 'req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.704700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77968fc2b85349b2a625ea9bf25cd026', 'request_id': 'req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.714171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77968fc2b85349b2a625ea9bf25cd026', 'request_id': 'req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.842634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] The token cache is being invalidate because project 77968fc2b85349b2a625ea9bf25cd026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:15:39.843252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.844484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-98e7c876-c6f7-4c76-8e68-a8ea6c929ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:39.847312 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1139/2293] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:15:39 2022] DELETE /identity/v3/projects/77968fc2b85349b2a625ea9bf25cd026 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:39.853704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:39.856028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:39.880620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b246887-9345-44e8-978c-ad159ac94c03 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:39.970533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:39.971081 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:39.971360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:39.971808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] PATH_INFO: `/v3/users/a6345ea169d248a89795ae42cd3f7df9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:39.972961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6345ea169d248a89795ae42cd3f7df9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:39.974926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:40.067026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:40.067464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:40.067887 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:40.090486 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:40.091773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:40.160040 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1140/2294] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:15:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:15:40.176072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:40.177623 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:40.231560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.249042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6345ea169d248a89795ae42cd3f7df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:40.249950 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6345ea169d248a89795ae42cd3f7df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:40.258880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6345ea169d248a89795ae42cd3f7df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:40.259750 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6345ea169d248a89795ae42cd3f7df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:40.274643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd400bd-cfae-481e-985e-898a1259062d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:40.279130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0fbc84e0-f878-4d93-bf8d-3d799587c525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6345ea169d248a89795ae42cd3f7df9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:40.281473 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.281857 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:40.282195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:40.282516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:40.282795 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:40.285215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:40.299802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:40.301494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:40.312421 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1141/2295] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:15:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:40.433464 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1155/2296] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:15:39 2022] DELETE /identity/v3/users/a6345ea169d248a89795ae42cd3f7df9 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:40.964621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:40.981578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:41.038983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb099e-23b1-4036-bf91-b915198ae654 service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:41.223832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:41.224542 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:41.224824 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:41.225102 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] PATH_INFO: `/v3/projects/860ce309152043c7b57fc66f1b3bb858` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:41.228966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860ce309152043c7b57fc66f1b3bb858)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:41.230993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:41.426878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860ce309152043c7b57fc66f1b3bb858', 'request_id': 'req-6c6ca937-bf04-417c-9b2c-9d6cb6325890'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:41.434640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860ce309152043c7b57fc66f1b3bb858', 'request_id': 'req-6c6ca937-bf04-417c-9b2c-9d6cb6325890'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:41.446902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860ce309152043c7b57fc66f1b3bb858', 'request_id': 'req-6c6ca937-bf04-417c-9b2c-9d6cb6325890'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:41.541118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] The token cache is being invalidate because project 860ce309152043c7b57fc66f1b3bb858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:15:41.541523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:41.542488 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-6c6ca937-bf04-417c-9b2c-9d6cb6325890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:15:41.545111 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1142/2297] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:15:40 2022] DELETE /identity/v3/projects/860ce309152043c7b57fc66f1b3bb858 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:15:41.554302 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:41.555249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:41.555249 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:41.590564 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:41.592625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:41.732445 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1156/2298] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:41.736036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:41.736478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:41.736942 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:41.766676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:41.768647 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:41.782628 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1143/2299] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:41.791637 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:41.814518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6df9d-ccda-4d87-b26c-ae3d67a70526 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:41.827523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:41.828417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:41.828714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:41.828989 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:41.830366 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:15:41.830688 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:41.833030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:41.853472 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1157/2300] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:15:41.855636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:41.872232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a50a7-adf7-419b-b8fe-5c914aa9072d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:41.881443 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:41.882506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:41.883001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:41.884107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:41.885327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:41.889150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.016060 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1144/2301] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:15:41 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:15:42.018193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:42.030064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cd6b5-8c17-4138-bef6-cdaf7bac77d6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:42.120230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:42.121023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.121379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.121730 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.122495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:42.124733 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.258709 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1158/2302] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:15:42 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:15:42.261139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:42.276531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2de2-0445-4cf0-b182-8dc8e2e06677 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:42.295968 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:42.297917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.298763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.301287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.304577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:42.309324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.346246 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1145/2303] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:15:42.346737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:42.359758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3275-271c-4965-a2ae-549bdeafca53 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:42.367522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:42.368604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.368986 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.369820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.370749 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:42.372885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d8ec2-a8b3-4ee1-8dcd-f7db1d443622 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.390762 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1159/2304] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:15:42.397081 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:42.422508 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506280-afde-4dc4-852e-bfaf87448c47 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:42.429976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:42.431989 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.435976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.435976 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] PATH_INFO: `/v3/projects/55f0064a8bc14f30b2749047dfd2f53d/users/4af60a16a7c24542b5b8aa94d2bfe9ef/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.480123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f0064a8bc14f30b2749047dfd2f53d, user_id=4af60a16a7c24542b5b8aa94d2bfe9ef, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:42.486450 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.492227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:42.585511 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1146/2305] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:15:42 2022] PUT /identity/v3/projects/55f0064a8bc14f30b2749047dfd2f53d/users/4af60a16a7c24542b5b8aa94d2bfe9ef/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:42.589085 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aff28-b225-48a0-8e40-ff87d1ff6b28 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.589933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aff28-b225-48a0-8e40-ff87d1ff6b28 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.589933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aff28-b225-48a0-8e40-ff87d1ff6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.626892 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-667aff28-b225-48a0-8e40-ff87d1ff6b28 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:42.628268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-667aff28-b225-48a0-8e40-ff87d1ff6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:42.761704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:42.772726 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1160/2306] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:15:42.790342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:42.792173 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:42.865370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28bff-3a03-49da-bc84-9943c52e2619 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:42.870233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:42.871032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:42.872930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:42.872930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:42.883946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:42.885394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:42.897932 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1147/2307] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:15:52.622830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:52.623424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:52.623424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:52.693628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] MFA Rules not processed for user `4af60a16a7c24542b5b8aa94d2bfe9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:15:52.696669 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:15:52.873726 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1161/2308] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:15:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 30 01:15:52.881197 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:52.900264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:52.907517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c1394-b1dc-4334-a590-4692a0719085 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:52.913322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:52.914010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:52.914302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:52.914577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:52.915245 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:52.917564 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.040525 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1148/2309] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:15:52 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:15:53.041661 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:53.055857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e760-5278-4fa4-9360-911a9f3e0442 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:53.149803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:53.150569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:53.150856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:53.151131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:53.151894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:53.153853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.316181 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1162/2310] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:15:53 2022] POST /identity/v3/users => generated 292 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:15:53.321305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:53.339386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb69250-f32c-4445-a109-432a2539388f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:53.349366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:53.350563 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:53.351074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:53.351614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:53.353020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:53.356727 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.374770 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1149/2311] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:15:53.394433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:53.394433 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d435b-0b8d-407f-9b21-d633fb9c608e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:53.402621 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:53.403457 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:53.403782 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:53.404477 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] PATH_INFO: `/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f/users/c5cb5f5e3fdb46169809497665ef4875/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:53.429728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8256fc8c274c61a65e34d10a6bf30f, user_id=c5cb5f5e3fdb46169809497665ef4875, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:53.433668 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.783002 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1163/2312] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:15:53 2022] PUT /identity/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f/users/c5cb5f5e3fdb46169809497665ef4875/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:15:53.795306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:53.799696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e034-5477-4203-af15-1bf27537ba80 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:53.881582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:53.882394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:53.882394 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:53.882733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:53.883941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:53.885614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2685ec-66bf-4d85-8023-e69684a0266a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.896920 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1150/2313] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:15:53.908604 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:15:53.922345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ad36-2efc-4157-8d34-15f3536f2d18 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:15:53.928319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:15:53.935139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:15:53.935139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:15:53.935139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] PATH_INFO: `/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f/users/c5cb5f5e3fdb46169809497665ef4875/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:15:53.937606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8256fc8c274c61a65e34d10a6bf30f, user_id=c5cb5f5e3fdb46169809497665ef4875, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:15:53.940033 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:15:53.954420 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1164/2314] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:15:53 2022] PUT /identity/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f/users/c5cb5f5e3fdb46169809497665ef4875/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:16:02.617410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:02.618930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:02.619404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:02.657518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] MFA Rules not processed for user `c5cb5f5e3fdb46169809497665ef4875`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:02.658959 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:02.772498 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1151/2315] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:16:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:02.790005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:02.791929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:02.851947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:02.900591 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4d3-92d4-4168-8379-be99611bc514 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:02.910283 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:02.911382 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:02.912066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:02.912586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:03.009523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:03.019434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064956d-ba26-42e5-8f67-ecf05cc53529 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:03.030146 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1165/2316] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:16:03.144043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:03.145837 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:03.211504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:03.215560 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a6a0d-ab32-4fbe-9661-e087188bc7d7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:03.219612 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:03.220272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:03.220555 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:03.220832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:03.222707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:03.243028 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:03.244797 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70977d1-9aeb-4b2a-a8b0-08a555ec607d service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:03.254778 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1152/2317] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:16:03.896140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:03.897193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:03.898396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:03.900281 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1166/2318] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:16:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:16:03.911419 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fc5f8-106c-4ec7-b37f-9e1f99e7fac0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:03.912108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fc5f8-106c-4ec7-b37f-9e1f99e7fac0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:03.912461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fc5f8-106c-4ec7-b37f-9e1f99e7fac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:03.946099 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4e3fc5f8-106c-4ec7-b37f-9e1f99e7fac0 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:03.948953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4e3fc5f8-106c-4ec7-b37f-9e1f99e7fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:03.967549 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1153/2319] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:16:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:03.988252 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:03.991718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:04.074906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36441681-76a2-41a1-b337-a3e60eed3fa1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:04.079030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:04.079974 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.080254 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.080533 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.093823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:04.095640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691a8a-5da2-460e-a46b-f80082ca98f9 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:04.114119 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1167/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:16:04.402455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633bb2-f8a5-4074-b558-751a1335aa35 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.403371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633bb2-f8a5-4074-b558-751a1335aa35 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.404130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633bb2-f8a5-4074-b558-751a1335aa35 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.407219 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1154/2321] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:16:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:16:04.414411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600cc67-b8c9-4995-ad91-077cd4569e99 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.414963 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600cc67-b8c9-4995-ad91-077cd4569e99 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.415368 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600cc67-b8c9-4995-ad91-077cd4569e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.456804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0600cc67-b8c9-4995-ad91-077cd4569e99 None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:04.458956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0600cc67-b8c9-4995-ad91-077cd4569e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:04.471215 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1168/2322] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:16:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:16:04.488911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39633bb2-f8a5-4074-b558-751a1335aa35 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:04.502992 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39633bb2-f8a5-4074-b558-751a1335aa35 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:04.528154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:04.528845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.529129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.529411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.542481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:04.544001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584a868-aa80-484a-aba3-538dca7a684d service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:04.553625 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1155/2323] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:16:04.676030 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9452-86d9-45e4-8abc-d95f29de4f96 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.677019 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9452-86d9-45e4-8abc-d95f29de4f96 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.678274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9452-86d9-45e4-8abc-d95f29de4f96 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.681926 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1169/2324] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:16:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:16:04.689811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b69b9-5bc8-47de-abc2-ee1c0104ab22 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.690445 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b69b9-5bc8-47de-abc2-ee1c0104ab22 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.691033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b69b9-5bc8-47de-abc2-ee1c0104ab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.725339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-a33b69b9-5bc8-47de-abc2-ee1c0104ab22 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:04.728011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-a33b69b9-5bc8-47de-abc2-ee1c0104ab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:04.746303 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1156/2325] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:16:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:16:04.764374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b9452-86d9-45e4-8abc-d95f29de4f96 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:04.784177 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b9452-86d9-45e4-8abc-d95f29de4f96 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:04.790608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:04.791413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:04.791981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:04.792312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:04.808226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:04.810104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253b473-37d4-46bf-8301-4186942c20d7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:04.823455 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1170/2326] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:16:07.001039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:07.002192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:07.002192 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:07.024516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] MFA Rules not processed for user `4af60a16a7c24542b5b8aa94d2bfe9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:07.025771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:07.035173 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1157/2327] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:16:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:10.103781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1f78a-708b-42f9-988a-1c953e498870 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:10.104234 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1f78a-708b-42f9-988a-1c953e498870 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:10.104643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1f78a-708b-42f9-988a-1c953e498870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:10.141663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4fd1f78a-708b-42f9-988a-1c953e498870 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:10.142938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4fd1f78a-708b-42f9-988a-1c953e498870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:10.213091 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1171/2328] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:16:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:10.229575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:10.229575 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:10.320228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4072b9-ee9a-404b-9f50-7707bb08eaec None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:10.326222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:10.326938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:10.327187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:10.327459 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:10.338846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:10.340644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:10.350535 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1158/2329] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:16:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:16:20.205169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:20.346161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=51 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:27.659955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd04c-ce64-45a7-98a4-f21ebba21af2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:27.661732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd04c-ce64-45a7-98a4-f21ebba21af2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:27.662630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd04c-ce64-45a7-98a4-f21ebba21af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:27.698939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0ffbd04c-ce64-45a7-98a4-f21ebba21af2 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:27.700870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0ffbd04c-ce64-45a7-98a4-f21ebba21af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:27.713934 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1172/2330] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:16:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:27.759364 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:27.776767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7e88-25b5-4904-879c-d8e4d95f5a6d service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:27.787275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:27.787275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:27.787275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:27.787275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:27.797835 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:27.799828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:27.814659 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1159/2331] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:16:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:16:28.279052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4862eb2-ad8f-4f86-8923-51b44755da56 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:28.279052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4862eb2-ad8f-4f86-8923-51b44755da56 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:28.279052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4862eb2-ad8f-4f86-8923-51b44755da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:28.313061 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c4862eb2-ad8f-4f86-8923-51b44755da56 None None] MFA Rules not processed for user `d09696979ec94a2699d7cb1e2aa77a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:16:28.315285 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c4862eb2-ad8f-4f86-8923-51b44755da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:16:28.400907 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1173/2332] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:16:37.809091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:38.392296 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=28 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:16:39.988430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:16:39.999500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f34a25-8416-4662-9616-2b7f23616913 service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:16:40.003874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:16:40.004524 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:16:40.004809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:16:40.005089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:16:40.015752 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:16:40.017276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc8383-6e3c-409b-8ace-296f53510198 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:16:40.029410 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1160/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:16:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:16:50.024447 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:17:16.132807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:17:16.133576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:17:16.133576 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:17:16.166898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] MFA Rules not processed for user `1d21128dee354a04bfb008efe6b60899`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:17:16.168253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:17:16.251692 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1174/2334] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:17:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:17:26.249804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=37 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:17:41.773370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ef0f-074a-4e78-9ec5-a576a0dad890 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:17:41.773997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ef0f-074a-4e78-9ec5-a576a0dad890 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:17:41.773997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ef0f-074a-4e78-9ec5-a576a0dad890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:17:41.795148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-36d3ef0f-074a-4e78-9ec5-a576a0dad890 None None] MFA Rules not processed for user `d09696979ec94a2699d7cb1e2aa77a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:17:41.796570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-36d3ef0f-074a-4e78-9ec5-a576a0dad890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:17:41.804275 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1161/2335] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:17:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:17:51.789659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:18:16.715126 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:16.716682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:16.839321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d398e04-2fd1-4f8b-9e6a-1aa515d04d7d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:16.846092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:16.846617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:16.846895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:16.847163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:16.849005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:16.863347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:16.865633 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:16.876456 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1175/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:18:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:18:26.175074 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f24a5-f88f-4f2f-ac70-9edaed5880ee None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:26.176094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f24a5-f88f-4f2f-ac70-9edaed5880ee None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:26.176094 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f24a5-f88f-4f2f-ac70-9edaed5880ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:26.218586 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-637f24a5-f88f-4f2f-ac70-9edaed5880ee None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:26.219902 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-637f24a5-f88f-4f2f-ac70-9edaed5880ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:26.236544 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1162/2337] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:18:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:18:26.383502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:26.396105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dcb9f-a53d-4b86-9b4e-681f68bb0755 service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:26.401561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:26.402223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:26.402479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:26.402753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:26.412763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:26.416841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b812-c18c-4900-a9d8-fb4638bfebfb service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:26.426541 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1176/2338] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:18:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:18:36.229644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=10 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:18:36.420536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=57 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:18:45.410153 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:45.411071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:45.411289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:45.413262 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1163/2339] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:18:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:18:45.421148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88167305-9492-4fb6-9734-43ed2bf79e77 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:45.422594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88167305-9492-4fb6-9734-43ed2bf79e77 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:45.423297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88167305-9492-4fb6-9734-43ed2bf79e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:45.484930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-88167305-9492-4fb6-9734-43ed2bf79e77 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:45.486323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-88167305-9492-4fb6-9734-43ed2bf79e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:45.512671 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1177/2340] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:18:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:18:45.529268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:45.532327 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:45.554198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b150eae-4622-4e1d-af28-8e88d1a0a9a1 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:45.558751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:45.559629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:45.560149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:45.560409 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:45.572971 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:45.574577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164231bb-9e58-4a40-aac7-816389ab27bc service glance] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:45.589408 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1164/2341] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:18:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:18:45.968769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70faf63-9b3e-4453-8789-fa8f0c2929e0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:45.969152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70faf63-9b3e-4453-8789-fa8f0c2929e0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:45.969475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70faf63-9b3e-4453-8789-fa8f0c2929e0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:45.971021 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1178/2342] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:18:45.977924 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88ba-588f-4cb9-88de-38856199c70c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:45.978425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88ba-588f-4cb9-88de-38856199c70c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:45.978872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88ba-588f-4cb9-88de-38856199c70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:46.018236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-168e88ba-588f-4cb9-88de-38856199c70c None None] MFA Rules not processed for user `c6194924f3054c9598c3548d8e87b294`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:46.019507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-168e88ba-588f-4cb9-88de-38856199c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:46.039383 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1165/2343] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:18:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:18:46.054323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70faf63-9b3e-4453-8789-fa8f0c2929e0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:46.066297 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70faf63-9b3e-4453-8789-fa8f0c2929e0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:46.071083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:46.072122 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:46.072514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:46.072841 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:46.082610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:46.084228 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0aa64-582c-4d72-b6f1-c23d0b9b7e09 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:46.097713 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1179/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:18:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:18:46.196720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b26da-fa5e-497b-8396-24a71c36a6b7 None None] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:46.197056 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b26da-fa5e-497b-8396-24a71c36a6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:46.197342 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b26da-fa5e-497b-8396-24a71c36a6b7 None None] PATH_INFO: `/` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:46.198795 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1166/2345] 10.222.0.26 () {54 vars in 921 bytes} [Sun Oct 30 01:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 01:18:46.206042 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7506dd-526c-4c1e-bd8e-c7b8eef24ef7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:46.206338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7506dd-526c-4c1e-bd8e-c7b8eef24ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:46.206592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7506dd-526c-4c1e-bd8e-c7b8eef24ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:46.231976 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8c7506dd-526c-4c1e-bd8e-c7b8eef24ef7 None None] MFA Rules not processed for user `eacd66576ed2425a8ad1fe0b69a4c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:46.233374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c7506dd-526c-4c1e-bd8e-c7b8eef24ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:46.244028 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1180/2346] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Oct 30 01:18:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:18:46.258185 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b26da-fa5e-497b-8396-24a71c36a6b7 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:46.271972 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b26da-fa5e-497b-8396-24a71c36a6b7 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:46.275937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:46.276569 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:46.276853 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:46.277195 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:46.289300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:46.290808 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:46.303881 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1167/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:18:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:18:49.472301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ceca1-98e3-4e5c-8812-5bc5c9dd7ca9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:49.473027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ceca1-98e3-4e5c-8812-5bc5c9dd7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:49.473334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ceca1-98e3-4e5c-8812-5bc5c9dd7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:49.509801 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ba9ceca1-98e3-4e5c-8812-5bc5c9dd7ca9 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:49.512678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ba9ceca1-98e3-4e5c-8812-5bc5c9dd7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:49.530261 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1181/2348] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:18:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:18:49.548023 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:18:49.561471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44091-069a-46ed-9ed3-e2026fc39ee3 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:18:49.594045 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:18:49.595428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:49.595918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:49.596395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:49.613072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:18:49.616417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9bd2b-0ea8-43c3-b780-9796c20942c2 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:18:49.627974 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1168/2349] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:18:54.236922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:18:54.243911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:18:54.243911 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:18:54.290163 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] MFA Rules not processed for user `d09696979ec94a2699d7cb1e2aa77a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:18:54.292142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:18:54.303397 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1182/2350] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:18:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:18:59.622927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=30 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:04.276114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=31 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:04.946027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:04.947022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:04.947300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:04.974317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:19:04.974317 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:04.981903 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1169/2351] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:19:04.996746 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:05.011793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a957f-8eef-4ea2-9eff-232bd726001c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:05.017007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:05.017630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:05.017916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:05.018192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:05.027962 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:05.029569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:05.041470 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1183/2352] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:19:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:19:09.894751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:09.894751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb49b1-068e-4eca-a2ae-7d5eb85d617a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:09.894751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:09.894751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:09.896901 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1170/2353] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:19:11.044066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:11.054347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830789ae-032b-4f9b-96bc-b8bbe5e54f73 service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:11.058220 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:11.058829 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:11.059113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:11.059414 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:11.061268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:11.075844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:11.077314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:11.089169 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1184/2354] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:19:19.849446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=9 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:21.083744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=12 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:41.029864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:41.031264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:41.112730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123be9-9e23-47cc-ac68-0cfd50aa4647 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:41.116585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:41.117128 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:41.117410 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:41.117687 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:41.119437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:41.132483 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:41.133926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:41.143409 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1171/2355] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:19:50.130261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:50.139573 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553bbb-3945-4680-8483-53d9eeb5f26b service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:50.144005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:50.144608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:50.144885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:50.145156 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:50.146906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:50.231567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:50.233051 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:50.251668 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1185/2356] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:19:51.139093 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=35 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:53.611304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:53.625585 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fe038-3d83-47b2-ad49-868caaa8327f service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:53.641539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:53.642562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:53.643682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:53.643965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:53.646151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:53.665061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:53.665061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:53.684769 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1172/2357] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:19:53.788553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:53.790056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:53.803611 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1c21-cc0a-4f4a-8913-84e33778bc3f service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:53.808408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:53.809076 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:53.809371 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:53.809627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] PATH_INFO: `/v3/users/1d21128dee354a04bfb008efe6b60899` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:53.810758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d21128dee354a04bfb008efe6b60899)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:53.812732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:54.023788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d21128dee354a04bfb008efe6b60899'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.024461 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d21128dee354a04bfb008efe6b60899'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.031560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d21128dee354a04bfb008efe6b60899'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.032187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d21128dee354a04bfb008efe6b60899'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.044246 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ead815e1-d821-4ecc-b63d-8fe268523dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d21128dee354a04bfb008efe6b60899'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.121570 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1186/2358] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:19:53 2022] DELETE /identity/v3/users/1d21128dee354a04bfb008efe6b60899 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:19:54.363500 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:54.381927 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e451c6e-db1b-42f4-bd32-797f4495c853 service placement] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:54.452020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:54.452667 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:54.452957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:54.453320 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] PATH_INFO: `/v3/projects/6e74d6179e574138841007195fd72400` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:54.454499 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e74d6179e574138841007195fd72400)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:54.456477 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:54.575460 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e74d6179e574138841007195fd72400', 'request_id': 'req-73e0f0e8-0060-4565-a413-fae6d854ceae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.587995 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e74d6179e574138841007195fd72400', 'request_id': 'req-73e0f0e8-0060-4565-a413-fae6d854ceae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.596326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e74d6179e574138841007195fd72400', 'request_id': 'req-73e0f0e8-0060-4565-a413-fae6d854ceae'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.718226 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] The token cache is being invalidate because project 6e74d6179e574138841007195fd72400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:19:54.718938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.719293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-73e0f0e8-0060-4565-a413-fae6d854ceae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:19:54.720863 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1173/2359] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:19:54 2022] DELETE /identity/v3/projects/6e74d6179e574138841007195fd72400 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:19:54.726101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:54.726372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:54.726647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:54.781876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:19:54.783115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:54.793360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:54.847880 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1187/2360] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:19:54.856478 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:54.856923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:54.857207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:54.880860 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:19:54.882250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:54.890432 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1174/2361] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:19:54.895752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:54.906602 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cd43b-c693-4134-b3d6-2e99369016df None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:54.911495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:54.912939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:54.912939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:54.912939 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:54.913337 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:19:54.913644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:54.915263 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:54.927202 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1188/2362] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:19:54.931870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:54.941809 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d518e8-8a89-4b8e-be46-efd167c149b8 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:54.950719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:54.950719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:54.950719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:54.950719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:54.951124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:54.951124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.075505 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1175/2363] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:19:54 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:19:55.079622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.090207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c1d1a-c654-4eb6-9776-7cda3618afaa admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.152651 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.153274 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.153563 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.153843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.154492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.156603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.261881 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1189/2364] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:19:55 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:19:55.266952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.278180 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2931-6578-4df9-9850-cdc9df7bdec0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.282969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.283604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.283895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.284173 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.284882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.286482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.297046 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1176/2365] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:19:55.301229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.311906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698fc6a-96b9-4966-8079-318b9af2ede9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.316871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.321814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.321814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.321814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.321814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.321814 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444355f-b299-40d2-a56a-45af25122af3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.332434 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1190/2366] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:19:55.336750 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.350264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2193fd-9c6b-437c-b644-e126372e2b62 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.353605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.354267 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.354546 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.354821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] PATH_INFO: `/v3/projects/cac80491a05545d5bc7a86ea1d8c44f2/users/4b26a80ce0494a2589e8c2cc5f719fe8/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.371510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac80491a05545d5bc7a86ea1d8c44f2, user_id=4b26a80ce0494a2589e8c2cc5f719fe8, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.374227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.503187 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1177/2367] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:19:55 2022] PUT /identity/v3/projects/cac80491a05545d5bc7a86ea1d8c44f2/users/4b26a80ce0494a2589e8c2cc5f719fe8/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:19:55.514451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.514918 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.515306 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.545747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] MFA Rules not processed for user `4b26a80ce0494a2589e8c2cc5f719fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:19:55.546996 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:55.621179 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1191/2368] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:19:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:19:55.628864 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.638132 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:55.707911 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c69e4-f933-41b1-a057-d1adbf83b83b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.714348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.714978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.715269 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.715551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.716328 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.718296 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:55.826680 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1178/2369] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:19:55 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:19:55.827896 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:55.839863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75312973-cb82-443a-ae3e-9a39092c89fe None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:55.849347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:55.903644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:55.904243 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:55.904698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:55.904992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:55.905662 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:55.907774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.005516 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1192/2370] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:19:55 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:19:56.009601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.020915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9070db-f582-4591-925a-4f9ec422f47b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.025966 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.026599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.026882 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.027162 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.027891 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.029831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.044160 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1179/2371] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:19:56.047546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.057983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e5796-3444-4b3e-b4be-a76a8addf19f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.062751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.063425 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.063757 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.064040 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] PATH_INFO: `/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5/users/e032b52de0994964adb8c14629bf1d44/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.080216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f542ea587444fca0c9e2e7fd9d2dc5, user_id=e032b52de0994964adb8c14629bf1d44, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.083954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.228353 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1193/2372] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:19:56 2022] PUT /identity/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5/users/e032b52de0994964adb8c14629bf1d44/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:19:56.232541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.242558 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c940c-dd7d-4571-a66f-9cb24e9b1b59 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.301937 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.302556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.302912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.303142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.303862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.305545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d9e51-b45e-4585-9c30-4c2a9999a7b0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.315927 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1180/2373] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:19:56.320831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.332115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acafa4-a8e6-43fe-8300-869d74160725 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.343953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.344678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.344988 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.345885 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] PATH_INFO: `/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5/users/e032b52de0994964adb8c14629bf1d44/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.348053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f542ea587444fca0c9e2e7fd9d2dc5, user_id=e032b52de0994964adb8c14629bf1d44, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.350817 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.362526 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1194/2374] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:19:56 2022] PUT /identity/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5/users/e032b52de0994964adb8c14629bf1d44/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:19:56.374630 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.375002 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.375341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.406798 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] MFA Rules not processed for user `e032b52de0994964adb8c14629bf1d44`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:19:56.408302 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:56.484461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:19:56.488821 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1181/2375] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:19:56.500703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.503788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:19:56.584288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286ff0c-2004-4528-806e-91700147ce1f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.592064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.592698 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.592983 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.593265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.607187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.608755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.620277 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1195/2376] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:19:56.795179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:19:56.809225 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ef48a-d685-4a95-a91a-afb06e0a9d48 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:19:56.815933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:19:56.815933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:19:56.815933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:19:56.815933 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:19:56.879769 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:19:56.881237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:19:56.892516 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1182/2377] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:19:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:02.468575 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:02.470816 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:02.580188 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:02.586104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca0cb1-6292-464d-8c06-e97436b5a420 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:02.592955 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:02.594599 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:02.594599 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:02.594599 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:02.596379 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:02.620694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:02.623184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:02.635603 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1196/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:06.888075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=28 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:09.799996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:09.801441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:09.861894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1010-7bea-4256-9840-a5943d500126 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:09.866150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:09.867090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:09.867377 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:09.867804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:09.869825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:09.938138 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:09.939693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:09.950309 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1183/2379] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:13.077622 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:13.604142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:13.608763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:13.685802 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1f416-2287-4285-a0a5-1ac5063f6b98 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:13.690812 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:13.691480 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:13.691793 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:13.692078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:13.693845 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:13.705874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:13.707406 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:13.718452 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1197/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:13.754048 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:13.756708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:13.780053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca381f-f5bc-456e-9959-da8e6ca28af0 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:13.785794 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:13.786441 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:13.786722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:13.787000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] PATH_INFO: `/v3/users/d09696979ec94a2699d7cb1e2aa77a64` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:13.788281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d09696979ec94a2699d7cb1e2aa77a64)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:13.791113 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:14.022089 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd09696979ec94a2699d7cb1e2aa77a64'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.022812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd09696979ec94a2699d7cb1e2aa77a64'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.035258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd09696979ec94a2699d7cb1e2aa77a64'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.123289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd09696979ec94a2699d7cb1e2aa77a64'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.130836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd09696979ec94a2699d7cb1e2aa77a64'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.137700 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1184/2381] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:13 2022] DELETE /identity/v3/users/d09696979ec94a2699d7cb1e2aa77a64 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:14.384863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:14.386582 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:14.401578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4af89-26a3-4070-8bfa-d1a54e3a3611 service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:14.405620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:14.407251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:14.442399 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:14.455392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:14.456052 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.456339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.456616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.458551 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:14.469202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:14.470687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde56094-324f-4a6d-a231-1a672146d969 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:14.484909 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33c791-fbf8-4151-8dcb-8dbe5ee71bc2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:14.489246 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1198/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:14.489950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:14.490773 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.491065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.491343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] PATH_INFO: `/v3/projects/cd37394129e346a8804d667856309041` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.492594 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd37394129e346a8804d667856309041)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:14.494521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:14.644400 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd37394129e346a8804d667856309041', 'request_id': 'req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.651066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd37394129e346a8804d667856309041', 'request_id': 'req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.659256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd37394129e346a8804d667856309041', 'request_id': 'req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.736699 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] The token cache is being invalidate because project cd37394129e346a8804d667856309041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:14.736903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.737574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-147b58a0-56fc-4a6c-8690-3d3b0c6b7872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:14.739099 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1185/2383] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:14 2022] DELETE /identity/v3/projects/cd37394129e346a8804d667856309041 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:14.744232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.744552 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.744857 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.767336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:14.767336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:14.832333 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1199/2384] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:14.841139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.846842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.846842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.864707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:14.865815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:14.880242 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1186/2385] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:14.881553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:14.893153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa779e-b12c-4108-a61e-eff0f3860577 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:14.898116 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:14.898731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.899046 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.899910 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.900583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:14.900894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:14.902554 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:14.919491 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1200/2386] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:20:14.932258 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:14.942255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2d752-36eb-4684-af12-f27d9fbb34e4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:14.948867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:14.949735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:14.949865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:14.950461 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:14.951234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:14.953210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.045385 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1187/2387] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:14 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:15.049756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:15.060429 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c51c-24c3-4aa5-a31f-2d941e768cdb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:15.085144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:15.133245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:15.136603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.136603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.136603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.136603 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:15.137842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.238722 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1201/2388] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:15 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:15.243901 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:15.254090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5c1ee-1be2-42fa-8943-6b4ebd4b99a4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:15.260055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:15.260840 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.261324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.261778 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.262554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:15.264528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.274907 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1188/2389] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:15.279265 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:15.289744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067f7cf-304d-47dd-a2c7-c1b9028df8de admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:15.294520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:15.295111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.297240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.297240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.297240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:15.298138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59716ed-5021-46f3-84f3-10a30c71a6d8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.310310 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1202/2390] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:15.314523 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:15.325578 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689659-57f6-4c41-bf99-91492dcb6759 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:15.331249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:15.331388 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.331633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.331921 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] PATH_INFO: `/v3/projects/ff5adf813db14e478b171abbd8ba07a7/users/0a98ada8957c407583a39423e4c11bbc/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.346509 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5adf813db14e478b171abbd8ba07a7, user_id=0a98ada8957c407583a39423e4c11bbc, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:15.349289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.439758 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1189/2391] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:15 2022] PUT /identity/v3/projects/ff5adf813db14e478b171abbd8ba07a7/users/0a98ada8957c407583a39423e4c11bbc/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:15.451952 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.452469 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.452570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.480873 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] MFA Rules not processed for user `0a98ada8957c407583a39423e4c11bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:15.484925 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:15.549232 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1203/2392] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:15.561278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:15.563152 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:15.613228 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:15.638322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd864-7c69-4a07-9a0f-b03ea658129f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:15.644086 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:15.644708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:15.644993 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:15.645270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:15.654026 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:15.655771 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a123ba-fab2-44ea-9359-6f3b15221fb8 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:15.665962 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1190/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:22.005994 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:22.072162 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e4603-e6c1-4ab4-8e7b-aa8e471f0816 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:22.077303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:22.077965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:22.078247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:22.078523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] PATH_INFO: `/v3/users/0a98ada8957c407583a39423e4c11bbc` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:22.079674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a98ada8957c407583a39423e4c11bbc)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:22.081635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:22.325123 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:22.343181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a98ada8957c407583a39423e4c11bbc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:22.343181 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a98ada8957c407583a39423e4c11bbc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:22.346616 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a98ada8957c407583a39423e4c11bbc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:22.347264 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a98ada8957c407583a39423e4c11bbc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:22.360023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a98ada8957c407583a39423e4c11bbc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:22.447553 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1204/2394] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:22 2022] DELETE /identity/v3/users/0a98ada8957c407583a39423e4c11bbc => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:22.452331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:22.452816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:22.452946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:22.472116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:22.473353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:22.533186 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1191/2395] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:22.543691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:22.545191 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:22.604701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a5a7-d640-4bc0-8e7d-6dcfb7d06b7a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:22.609222 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:22.609834 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:22.610120 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:22.610395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:22.623140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:22.629173 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d566-0a86-4916-a5ca-0ca2a04fd28d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:22.635737 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1205/2396] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:23.102670 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.104260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:23.117260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c7a9a-7c19-4b6b-910a-9d9ba2cf15cc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.123300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.124103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.124305 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.129024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] PATH_INFO: `/v3/projects/ff5adf813db14e478b171abbd8ba07a7` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.129024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff5adf813db14e478b171abbd8ba07a7)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.129024 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.258015 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff5adf813db14e478b171abbd8ba07a7', 'request_id': 'req-ed02afe4-01f5-4438-a51c-1ea980a57517'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:23.263898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff5adf813db14e478b171abbd8ba07a7', 'request_id': 'req-ed02afe4-01f5-4438-a51c-1ea980a57517'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:23.270430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff5adf813db14e478b171abbd8ba07a7', 'request_id': 'req-ed02afe4-01f5-4438-a51c-1ea980a57517'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:23.380171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] The token cache is being invalidate because project ff5adf813db14e478b171abbd8ba07a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:23.380475 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:23.381117 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-ed02afe4-01f5-4438-a51c-1ea980a57517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:23.388039 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1192/2397] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:23 2022] DELETE /identity/v3/projects/ff5adf813db14e478b171abbd8ba07a7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:23.391742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.392207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.392325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.413006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:23.414376 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:23.476701 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1206/2398] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:23.486087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.486657 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.486965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.507843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:23.509137 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:23.517542 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1193/2399] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:23.521883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.533015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c3a05-0081-48f7-abbf-ca90042f580f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.537878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.538493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.538780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.539055 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.539889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:23.540203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.541803 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.557670 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1207/2400] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:20:23.558944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.575816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb674415-7c03-4f18-8801-c2d988e0949d None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.586235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.586867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.587148 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.587458 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.588131 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.590561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.700824 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1194/2401] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:23 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:23.705346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.716690 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ac95d-a62e-4620-b621-bffc84978a74 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.750415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:23.773822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.774505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.774796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.775077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.775752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.777725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.895348 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1208/2402] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:23 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:23.899671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.910739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193628e-dbd5-4c06-92b3-0d679bcbc6fd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.915482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.916135 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.916432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.916704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.917341 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.919078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.929448 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1195/2403] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:23.933423 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.943638 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99ee2-6383-4f1d-bf09-2f99c4f3c210 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.948559 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.949169 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.949476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.950268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.950966 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:23.952600 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfa0d0-b5e5-4c8c-a6ba-cb45586ab81f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:23.962962 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1209/2404] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:23.967708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:23.978210 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de858e-6c39-4654-b14f-a93a7147345b admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:23.982953 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:23.983619 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:23.983905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:23.984181 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] PATH_INFO: `/v3/projects/d9ba80c76926468a96822ba92a1f6f80/users/f142e6142c4d433f88de674a93e391c9/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:23.998870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ba80c76926468a96822ba92a1f6f80, user_id=f142e6142c4d433f88de674a93e391c9, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.001605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.003757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:24.101064 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1196/2405] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:23 2022] PUT /identity/v3/projects/d9ba80c76926468a96822ba92a1f6f80/users/f142e6142c4d433f88de674a93e391c9/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:24.111984 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.112275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.112560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.142193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] MFA Rules not processed for user `f142e6142c4d433f88de674a93e391c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:24.143574 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:24.210623 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1210/2406] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:24.215072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.278412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee75b08-4ac0-4e35-af68-73520371cfb4 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.283114 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.283744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.284025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.284348 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.284980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.286899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.399223 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1197/2407] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:24 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:24.403546 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.414783 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dddb4c-6b7b-4318-871d-0db19c9a5fe5 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.475266 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.475856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.476170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.476450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.477099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.479325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.621465 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1211/2408] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:24 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:24.625682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.636142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b3174-4c21-44d8-b803-aa23dda95d7a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.641413 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.642075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.642359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.642628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.643274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.644944 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.658518 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1198/2409] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:24.662718 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.673708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f21d50-d05d-4135-9bf4-bc194142e748 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.678920 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.679664 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.680895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.680895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] PATH_INFO: `/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be/users/01d3fd4068ae4820ae7284c931308c71/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.695608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f1d8b3f98f4f1cbca84e5bb70d94be, user_id=01d3fd4068ae4820ae7284c931308c71, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.698355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.821996 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1212/2410] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:24 2022] PUT /identity/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be/users/01d3fd4068ae4820ae7284c931308c71/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:24.826983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.837533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d22f4-66e0-42f3-bbc5-309a9fc88f6c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.895626 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.896143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.896427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.896708 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.897517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.899238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93164-d3af-49b4-84f4-ae8d1d204d2a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.911910 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1199/2411] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:24.923223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:24.935786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a00d-fa14-4ff0-b3ea-007fbde8123b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:24.940788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:24.941404 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.943646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.943646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] PATH_INFO: `/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be/users/01d3fd4068ae4820ae7284c931308c71/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:24.944070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f1d8b3f98f4f1cbca84e5bb70d94be, user_id=01d3fd4068ae4820ae7284c931308c71, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:24.946640 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:24.953242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:24.958620 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1213/2412] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:24 2022] PUT /identity/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be/users/01d3fd4068ae4820ae7284c931308c71/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:24.972526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:24.972526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:24.972526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:25.004236 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] MFA Rules not processed for user `01d3fd4068ae4820ae7284c931308c71`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:25.005526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:25.059861 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:25.081641 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1200/2413] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:25.091924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:25.093755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:25.155883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c4461-9601-4dd4-9165-a34ec1c84779 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:25.159567 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:25.160204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:25.160483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:25.160756 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:25.171108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:25.173168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9442-0644-4066-afd0-c13a74e980b1 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:25.183321 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1214/2414] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:26.249579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:26.314183 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876a42e-6bc6-43bf-b096-af75992573ce None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:26.318846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:26.319506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:26.319814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:26.320088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] PATH_INFO: `/v3/users/f142e6142c4d433f88de674a93e391c9` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:26.321216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f142e6142c4d433f88de674a93e391c9)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:26.323204 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:26.536897 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f142e6142c4d433f88de674a93e391c9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:26.537570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f142e6142c4d433f88de674a93e391c9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:26.550739 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f142e6142c4d433f88de674a93e391c9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:26.649329 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f142e6142c4d433f88de674a93e391c9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:26.658964 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f142e6142c4d433f88de674a93e391c9'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:26.666341 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1201/2415] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:26 2022] DELETE /identity/v3/users/f142e6142c4d433f88de674a93e391c9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:26.671884 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:26.672832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:26.673121 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:26.693279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:26.694497 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:26.753944 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1215/2416] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:26.768728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:26.768728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:26.829849 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ff67f-1f93-42ae-8ba7-f9f3b33ec905 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:26.833629 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:26.834255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:26.834567 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:26.834848 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:26.844168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:26.848554 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:26.855713 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1202/2417] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:27.325362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:27.326879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:27.341444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662edf13-1b09-431e-a700-b3f3352214ed None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:27.346183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:27.346827 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:27.347109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:27.347385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] PATH_INFO: `/v3/projects/d9ba80c76926468a96822ba92a1f6f80` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:27.348550 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ba80c76926468a96822ba92a1f6f80)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:27.350601 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:27.454998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ba80c76926468a96822ba92a1f6f80', 'request_id': 'req-a9b232b3-797c-4973-bc17-fe4b93d3ce81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.461130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ba80c76926468a96822ba92a1f6f80', 'request_id': 'req-a9b232b3-797c-4973-bc17-fe4b93d3ce81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.533387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ba80c76926468a96822ba92a1f6f80', 'request_id': 'req-a9b232b3-797c-4973-bc17-fe4b93d3ce81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.546459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] The token cache is being invalidate because project d9ba80c76926468a96822ba92a1f6f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:27.546769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.547402 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.548940 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1216/2418] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:27 2022] DELETE /identity/v3/projects/d9ba80c76926468a96822ba92a1f6f80 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:27.553171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:27.563238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a369-f259-4474-9d19-6247f05d35e6 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:27.571323 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:27.627875 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:27.628519 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:27.628802 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:27.629080 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] PATH_INFO: `/v3/users/01d3fd4068ae4820ae7284c931308c71` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:27.630235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d3fd4068ae4820ae7284c931308c71)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:27.632582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:27.850437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d3fd4068ae4820ae7284c931308c71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.856764 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d3fd4068ae4820ae7284c931308c71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.868736 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d3fd4068ae4820ae7284c931308c71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.940249 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d3fd4068ae4820ae7284c931308c71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.948780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-23eae24a-d230-44fe-903d-6b0fb96c13f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d3fd4068ae4820ae7284c931308c71'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:27.955207 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1203/2419] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:27 2022] DELETE /identity/v3/users/01d3fd4068ae4820ae7284c931308c71 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:28.471041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:28.472631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:28.488721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b232b3-797c-4973-bc17-fe4b93d3ce81 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:28.513004 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:28.553020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:28.560562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:28.560562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:28.560562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] PATH_INFO: `/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:28.560562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f1d8b3f98f4f1cbca84e5bb70d94be)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:28.560562 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:28.756832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f1d8b3f98f4f1cbca84e5bb70d94be', 'request_id': 'req-0bf40b56-eeea-4e14-b654-10388db16a3f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:28.762087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f1d8b3f98f4f1cbca84e5bb70d94be', 'request_id': 'req-0bf40b56-eeea-4e14-b654-10388db16a3f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:28.900270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f1d8b3f98f4f1cbca84e5bb70d94be', 'request_id': 'req-0bf40b56-eeea-4e14-b654-10388db16a3f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:28.913725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] The token cache is being invalidate because project b1f1d8b3f98f4f1cbca84e5bb70d94be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:28.913856 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:28.914487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0bf40b56-eeea-4e14-b654-10388db16a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:28.916058 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1217/2420] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:28 2022] DELETE /identity/v3/projects/b1f1d8b3f98f4f1cbca84e5bb70d94be => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:28.921793 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:28.922337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:28.922337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:28.942381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:28.943631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:29.005899 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1204/2421] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:29.014581 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.014870 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.015152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.036310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:29.037572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:29.045746 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1218/2422] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:29.050032 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.060720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9286fc-d4cb-449c-8f2f-d64c7cb353ff None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.066250 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.066943 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.067259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.067618 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.068502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:29.068836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.070611 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.084071 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1205/2423] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:20:29.088912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.103747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b95f3c-93e8-4a34-919c-e00c2f6b4022 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.106071 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.106691 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.106969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.107471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.107942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.109959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.216167 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1219/2424] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:29 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:29.220660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.235529 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a72-aeb1-4d85-bd31-41487d6762b6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.253742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:29.301622 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.302244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.302520 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.302804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.303768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.305533 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.431746 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1206/2425] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:29 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:29.436435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.449527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ecd16-8d25-408e-8766-13944ca6d7c5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.455094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.455791 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.456082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.456357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.461775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.461775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.470593 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1220/2426] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:29.477263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.496260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ba3d-0c71-4439-a959-ce85b7c8fba0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.501653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.502704 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.503186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.503707 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.504846 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.506969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703900ae-6bcc-4df5-93c3-2f4761e4456e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.520579 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1207/2427] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:29.526901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.540981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787292a4-7c3b-4bde-b630-6b0886bc604f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.546331 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.547262 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.547608 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.547985 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] PATH_INFO: `/v3/projects/8cc17d72cc154a6cbc74d6a82e551a44/users/2353fdbaf8dd4b1e99fc66e92a447873/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.566008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc17d72cc154a6cbc74d6a82e551a44, user_id=2353fdbaf8dd4b1e99fc66e92a447873, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.568914 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:29.645715 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1221/2428] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:29 2022] PUT /identity/v3/projects/8cc17d72cc154a6cbc74d6a82e551a44/users/2353fdbaf8dd4b1e99fc66e92a447873/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:29.660339 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.660784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.661178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.699347 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] MFA Rules not processed for user `2353fdbaf8dd4b1e99fc66e92a447873`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:29.700645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:29.781560 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1208/2429] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:29.784202 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:29.844754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:29.857557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e5f5-406d-49ea-8ee6-4e1dc0e3ecdd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:29.863395 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:29.863654 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:29.863924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:29.864270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:29.865313 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:29.867151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.148960 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1222/2430] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:29 2022] POST /identity/v3/projects => generated 369 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:30.153125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.164730 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4006290-9cde-4aea-abf3-3db7f0f539a4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.235562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.236304 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.236591 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.236870 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.237514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.239457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.394743 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1209/2431] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:30 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:30.399645 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.410953 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874a3d-325e-4c61-9abb-c1d1a2a92767 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.415697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.416315 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.416595 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.416867 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.417522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.419794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.430514 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1223/2432] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:30.436006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.447879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13572b-7d94-4242-be12-ad66008c64cd admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.452735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.453437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.453720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.453996 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] PATH_INFO: `/v3/projects/a589f1cec29c4330af91dcb0a5105c56/users/267932fa554a4c41b2cb644bb84c7290/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.467446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:30.470663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a589f1cec29c4330af91dcb0a5105c56, user_id=267932fa554a4c41b2cb644bb84c7290, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.473411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.575571 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1210/2433] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:30 2022] PUT /identity/v3/projects/a589f1cec29c4330af91dcb0a5105c56/users/267932fa554a4c41b2cb644bb84c7290/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:30.579692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.590658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a7b5d-4854-4140-8bb3-a5a6e4779ced admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.653093 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.653620 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.653898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.654176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.654864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.656479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165cea-4d67-4e55-8761-09e0cceaef92 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.670760 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1224/2434] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:30.680005 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.685553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1dccf-9250-45fc-8ce4-a142c036e740 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.690518 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.691198 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.691693 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.691979 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] PATH_INFO: `/v3/projects/a589f1cec29c4330af91dcb0a5105c56/users/267932fa554a4c41b2cb644bb84c7290/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.693935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a589f1cec29c4330af91dcb0a5105c56, user_id=267932fa554a4c41b2cb644bb84c7290, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.696828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.708120 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1211/2435] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:30 2022] PUT /identity/v3/projects/a589f1cec29c4330af91dcb0a5105c56/users/267932fa554a4c41b2cb644bb84c7290/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:30.724560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.724847 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.725125 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.757923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] MFA Rules not processed for user `267932fa554a4c41b2cb644bb84c7290`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:30.759164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:30.841455 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1225/2436] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:30.853743 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:30.855502 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:30.923457 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de20447-9ab8-4935-bfd1-ac95a3bde88f admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:30.927872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:30.928511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:30.928792 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:30.929075 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:30.939733 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:30.941239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:30.951911 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1212/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:31.124316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:31.126124 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:31.138928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c56d67-c1cd-43e5-9fca-2a1fd19a1961 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:31.142625 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:31.143241 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:31.143657 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:31.143945 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:31.195830 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:31.210230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:31.211751 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296ad72-e238-4fc5-ac91-9ccc2000d9ad service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:31.222358 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1226/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:35.824807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:35.826247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:35.900078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02822f3-be15-4418-be27-51ba15a99f1b service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:35.905514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:35.906780 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:35.907062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:35.907337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] PATH_INFO: `/v3/users/4b26a80ce0494a2589e8c2cc5f719fe8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:35.908726 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b26a80ce0494a2589e8c2cc5f719fe8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:35.910677 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:36.162678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b26a80ce0494a2589e8c2cc5f719fe8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.163472 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b26a80ce0494a2589e8c2cc5f719fe8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.177120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b26a80ce0494a2589e8c2cc5f719fe8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.331437 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b26a80ce0494a2589e8c2cc5f719fe8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.346082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b26a80ce0494a2589e8c2cc5f719fe8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.349155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:36.355742 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1213/2439] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:35 2022] DELETE /identity/v3/users/4b26a80ce0494a2589e8c2cc5f719fe8 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:36.362325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:36.363083 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:36.363536 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:36.386909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:36.388164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:36.457374 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1227/2440] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:36.476697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:36.478582 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:36.547562 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88985f83-f88e-4d27-9a2a-e4e907b1087d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:36.557259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:36.557259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:36.557259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:36.557259 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:36.564276 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:36.565757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d4742-58f4-4c1e-857f-72d763ac5d1e service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:36.576313 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1214/2441] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:36.620180 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:36.621745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:36.634539 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6625e-7799-400c-908f-d10cf9fb089d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:36.639702 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:36.640374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:36.640642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:36.640904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] PATH_INFO: `/v3/users/2353fdbaf8dd4b1e99fc66e92a447873` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:36.642109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2353fdbaf8dd4b1e99fc66e92a447873)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:36.644091 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:36.871271 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2353fdbaf8dd4b1e99fc66e92a447873'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.872351 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2353fdbaf8dd4b1e99fc66e92a447873'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.882703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2353fdbaf8dd4b1e99fc66e92a447873'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.883491 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2353fdbaf8dd4b1e99fc66e92a447873'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:36.907969 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2353fdbaf8dd4b1e99fc66e92a447873'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:37.012221 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1228/2442] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:36 2022] DELETE /identity/v3/users/2353fdbaf8dd4b1e99fc66e92a447873 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:37.016941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:37.017435 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:37.017553 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:37.037361 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:37.038888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:37.111846 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1215/2443] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:37.127928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:37.127928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:37.197947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f688-5209-4f87-93f7-fb0d64cd64a1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:37.202683 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:37.203352 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:37.203849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:37.204127 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:37.214523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:37.219752 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:37.222084 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:37.228064 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1229/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:37.644061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:37.645643 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:37.749422 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0b57a-7e63-4e03-996d-e195d2952384 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:37.775246 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:37.779232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:37.779696 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:37.781355 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:37.793208 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:37.834036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:37.835462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:37.864008 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1216/2445] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:37.945190 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:37.946170 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:37.964703 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843355e-3bba-40a8-a77b-ec1c51606cb6 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:37.970790 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:37.978762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:37.978762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:37.978762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] PATH_INFO: `/v3/projects/cac80491a05545d5bc7a86ea1d8c44f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:37.978762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac80491a05545d5bc7a86ea1d8c44f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:37.978762 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:38.006918 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:38.009963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:38.042433 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf66d8e-e5ec-49c4-b7f1-20ddbe9abdd0 service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:38.053196 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:38.054664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:38.055144 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:38.055692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] PATH_INFO: `/v3/projects/8cc17d72cc154a6cbc74d6a82e551a44` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:38.059973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc17d72cc154a6cbc74d6a82e551a44)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:38.066536 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:38.123934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:38.230697 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac80491a05545d5bc7a86ea1d8c44f2', 'request_id': 'req-dc32fd89-50eb-4768-83ef-f45ca3e9e049'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.241988 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac80491a05545d5bc7a86ea1d8c44f2', 'request_id': 'req-dc32fd89-50eb-4768-83ef-f45ca3e9e049'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.384146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac80491a05545d5bc7a86ea1d8c44f2', 'request_id': 'req-dc32fd89-50eb-4768-83ef-f45ca3e9e049'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.397205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] The token cache is being invalidate because project cac80491a05545d5bc7a86ea1d8c44f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:38.397527 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.398193 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.399755 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1230/2446] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:37 2022] DELETE /identity/v3/projects/cac80491a05545d5bc7a86ea1d8c44f2 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:38.403769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:38.405187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:38.435432 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc17d72cc154a6cbc74d6a82e551a44', 'request_id': 'req-2ebdf755-af95-4c10-8e8a-90440309cdfa'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.443171 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc17d72cc154a6cbc74d6a82e551a44', 'request_id': 'req-2ebdf755-af95-4c10-8e8a-90440309cdfa'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.452543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc17d72cc154a6cbc74d6a82e551a44', 'request_id': 'req-2ebdf755-af95-4c10-8e8a-90440309cdfa'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.491230 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fd89-50eb-4768-83ef-f45ca3e9e049 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:38.528745 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] The token cache is being invalidate because project 8cc17d72cc154a6cbc74d6a82e551a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:38.529176 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.531101 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.531661 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1217/2447] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:38 2022] DELETE /identity/v3/projects/8cc17d72cc154a6cbc74d6a82e551a44 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:38.535579 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:38.537053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:38.564312 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:38.564970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:38.565319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:38.565632 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] PATH_INFO: `/v3/users/e032b52de0994964adb8c14629bf1d44` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:38.566833 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e032b52de0994964adb8c14629bf1d44)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:38.568931 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:38.617427 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdf755-af95-4c10-8e8a-90440309cdfa admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:38.622187 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:38.622930 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:38.623217 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:38.623494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] PATH_INFO: `/v3/users/267932fa554a4c41b2cb644bb84c7290` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:38.624782 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] RBAC: Authorizing `identity:delete_user(user_id=267932fa554a4c41b2cb644bb84c7290)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:38.627278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:38.763663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:38.776464 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e032b52de0994964adb8c14629bf1d44'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.777185 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e032b52de0994964adb8c14629bf1d44'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.787090 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e032b52de0994964adb8c14629bf1d44'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.787882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e032b52de0994964adb8c14629bf1d44'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.803307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e032b52de0994964adb8c14629bf1d44'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.862497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '267932fa554a4c41b2cb644bb84c7290'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.863280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '267932fa554a4c41b2cb644bb84c7290'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.877360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '267932fa554a4c41b2cb644bb84c7290'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.882607 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1231/2448] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:38 2022] DELETE /identity/v3/users/e032b52de0994964adb8c14629bf1d44 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:38.957659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '267932fa554a4c41b2cb644bb84c7290'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.965874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '267932fa554a4c41b2cb644bb84c7290'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:38.972800 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1218/2449] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:38 2022] DELETE /identity/v3/users/267932fa554a4c41b2cb644bb84c7290 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:39.446715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:39.448407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.482981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:39.484756 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.547216 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:39.564298 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83707073-026d-474d-abef-222dd6278dfe admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:39.584899 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2635f51-9b33-4887-ad63-e93cb15a151c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:39.590396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:39.591293 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.591680 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.592036 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] PATH_INFO: `/v3/projects/a589f1cec29c4330af91dcb0a5105c56` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.593353 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a589f1cec29c4330af91dcb0a5105c56)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:39.597412 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:39.631203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:39.632082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.632192 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.632440 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] PATH_INFO: `/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.633643 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f542ea587444fca0c9e2e7fd9d2dc5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:39.636290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:39.717817 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a589f1cec29c4330af91dcb0a5105c56', 'request_id': 'req-49dca4dd-a096-424a-bb63-6ee6b0b1b808'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.724280 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a589f1cec29c4330af91dcb0a5105c56', 'request_id': 'req-49dca4dd-a096-424a-bb63-6ee6b0b1b808'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.731737 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a589f1cec29c4330af91dcb0a5105c56', 'request_id': 'req-49dca4dd-a096-424a-bb63-6ee6b0b1b808'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.749104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f542ea587444fca0c9e2e7fd9d2dc5', 'request_id': 'req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.756184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f542ea587444fca0c9e2e7fd9d2dc5', 'request_id': 'req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.757852 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:39.803438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] The token cache is being invalidate because project a589f1cec29c4330af91dcb0a5105c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:39.803786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.804454 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-49dca4dd-a096-424a-bb63-6ee6b0b1b808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.806069 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1219/2450] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:39 2022] DELETE /identity/v3/projects/a589f1cec29c4330af91dcb0a5105c56 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:39.810713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b06d-fdec-48cf-a20b-a8fe4c7baee2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.811034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b06d-fdec-48cf-a20b-a8fe4c7baee2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.811682 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b06d-fdec-48cf-a20b-a8fe4c7baee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.832855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6b06b06d-fdec-48cf-a20b-a8fe4c7baee2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:39.834073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6b06b06d-fdec-48cf-a20b-a8fe4c7baee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.872101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f542ea587444fca0c9e2e7fd9d2dc5', 'request_id': 'req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.891104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] The token cache is being invalidate because project 46f542ea587444fca0c9e2e7fd9d2dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:39.891424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.892103 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c8cb1d8e-ad67-45f4-8867-10571a5b4f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:39.894185 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1232/2451] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:39 2022] DELETE /identity/v3/projects/46f542ea587444fca0c9e2e7fd9d2dc5 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:39.898692 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ed59e-2718-4cbe-aab1-a11c5a35b091 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.898977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ed59e-2718-4cbe-aab1-a11c5a35b091 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.899259 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ed59e-2718-4cbe-aab1-a11c5a35b091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.907246 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1220/2452] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:39.917601 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.919229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.919229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.924465 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-491ed59e-2718-4cbe-aab1-a11c5a35b091 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:39.925978 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-491ed59e-2718-4cbe-aab1-a11c5a35b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.934501 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1233/2453] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:39.941463 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:39.945150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.945336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.945336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.945336 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.957934 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1221/2454] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:39.959033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:39.968977 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e22d0a-ba99-4753-b3eb-6d6989eb4bf4 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:39.971788 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:39.973183 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:39.974896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:39.975530 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:39.975830 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:39.976108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:39.976757 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:39.977067 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:39.979393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:39.983763 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1234/2455] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:39.991526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:39.995756 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1222/2456] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:20:40.000324 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.001938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6043a-d911-40ea-ab0f-ade68a34e4b7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.007578 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.014040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a02e9-fe21-4823-b4cb-382c16e97dc6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.014194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.014194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.014194 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.016050 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:40.016523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.016896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.017503 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.017832 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.018112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.018788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.019525 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.020740 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.041005 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1235/2457] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:20:40.042882 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.051992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f8de4-d021-43c1-8faa-c63468546844 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.056849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.057460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.057737 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.058032 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.058687 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.060623 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.142429 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1223/2458] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:39 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:40.146660 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.167889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:40.211308 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3a7e4-761c-48eb-9ac7-77bb96cfee64 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.216107 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1236/2459] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:40.216315 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.216725 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.217007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.217285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.217950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.220055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.231105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.323082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e64270-d7fc-4675-bed7-a3fc9ad680d4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.323082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.323082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.323082 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.323481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.323481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.323481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.348509 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1224/2460] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:40.352697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.363788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5bf44-c63d-4fa5-ba42-68014a107508 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.370713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.370713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.370713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.371238 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.371561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.373251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.388050 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1225/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:40.390111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.400625 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13b03-d814-47b4-a7c0-9acc2249c33c admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.406156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.406779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.407057 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.407331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.408014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.409645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.421364 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1226/2462] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:40.426058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.429370 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1237/2463] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:40.433496 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.438179 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ce2f7-6146-4673-8995-bc3f6b772cce admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.443205 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.443934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.444220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.444498 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] PATH_INFO: `/v3/projects/0f8fdca8399d4d3b8f7dac71e4db43cc/users/c2329d57275d4f1a999b2d05e0e36ec1/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.449871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a36e-2f87-4a90-9c78-df1ecd1829d3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.455532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.456245 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.456837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.456837 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.457474 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.459087 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.467452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8fdca8399d4d3b8f7dac71e4db43cc, user_id=c2329d57275d4f1a999b2d05e0e36ec1, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.471044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc6781-d6e9-482e-bc43-8ae6fd8f581a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.472026 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1238/2464] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:40.476146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.486684 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d527c9-8fc7-4a42-ae90-aafd5557bcb4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.492036 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.492642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.492923 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.493247 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.493889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.495514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.511258 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1239/2465] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:40.512534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.523481 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcabef-3fc8-4d8b-8e9f-1610680098a1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.528174 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.528863 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.529109 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.529386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] PATH_INFO: `/v3/projects/bbd9de5d7b7d4b518c27122aec803dba/users/94c64f76058e493d958ec3f96a56a0e0/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.544928 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd9de5d7b7d4b518c27122aec803dba, user_id=94c64f76058e493d958ec3f96a56a0e0, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.548175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29932b6-af4e-4a51-bfce-20db5a38d8bd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.597986 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1227/2466] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:40 2022] PUT /identity/v3/projects/0f8fdca8399d4d3b8f7dac71e4db43cc/users/c2329d57275d4f1a999b2d05e0e36ec1/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:40.636105 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1240/2467] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:40 2022] PUT /identity/v3/projects/bbd9de5d7b7d4b518c27122aec803dba/users/94c64f76058e493d958ec3f96a56a0e0/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:40.641001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.641325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.641570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.673523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.673886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.674184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.696496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] MFA Rules not processed for user `94c64f76058e493d958ec3f96a56a0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:40.697900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:40.711597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] MFA Rules not processed for user `c2329d57275d4f1a999b2d05e0e36ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:40.713706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:40.756026 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:40.829713 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1241/2468] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:40.837377 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.840534 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1228/2469] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:40.845345 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:40.900506 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:40.904922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3020fa-5790-4231-b8c9-d6ee688a23b2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.910275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.914039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.914039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.914039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.914039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.914442 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebad96-77ad-4eef-9551-44c0bad15a98 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:40.914542 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.917244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:40.917917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:40.918337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:40.918786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:40.919499 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:40.921526 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:40.997692 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1242/2470] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:41.001743 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.026665 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1229/2471] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:40 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:41.030903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.083961 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039a2c-10a7-4028-a90b-889044ac0047 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.102671 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951f514-4da4-4a08-a79e-b6f0b1e1b6d0 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.107791 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.110411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.110411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.110411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.110411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.111742 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.148667 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.149291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.149570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.149842 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.150483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.152781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.164635 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:41.213830 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1230/2472] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:41 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:41.215451 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.228001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc2ed2-2e7a-4e9d-bda3-d4d2f3397104 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.233973 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.234600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.234885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.235168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.236247 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.237544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.251049 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1243/2473] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:41 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:41.251317 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1231/2474] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:41.254175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.254462 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.266590 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b2f2-73f1-44ff-97ec-3efb20518c4a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.267106 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ef3c-3278-4988-8699-fcbe7d0c3871 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.271939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.272613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.272950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.273235 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] PATH_INFO: `/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb/users/e08be9d9d1d745a8932a73ccb58a9b34/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.273478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.274316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.274694 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.275113 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.276079 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.278307 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.290784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba2102cf0a14b0fbe3dfe794fbc2bbb, user_id=e08be9d9d1d745a8932a73ccb58a9b34, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.291950 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1244/2475] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:41.293535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.296460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.307357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc31e9e-9bce-44b3-aa55-b4aaa7481ba6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.312006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.319005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.319005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.319005 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] PATH_INFO: `/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f/users/d1cbeb06c3ca4ded9e3e060e349be2c8/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.330614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2083ae9a19c4575b2d29b4be9d01f0f, user_id=d1cbeb06c3ca4ded9e3e060e349be2c8, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.333615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.409353 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1232/2476] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:41 2022] PUT /identity/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb/users/e08be9d9d1d745a8932a73ccb58a9b34/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:41.413663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.436188 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1245/2477] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:41 2022] PUT /identity/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f/users/d1cbeb06c3ca4ded9e3e060e349be2c8/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:41.447822 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.490545 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f48fc-936a-4b50-aca1-58ce790efdc8 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.516943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc215263-7b61-4d06-842e-a723a4776f16 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.521922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.522544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.522840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.523115 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.523799 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.525413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.535962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:41.536452 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1246/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:41.543547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.558512 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb51e-1868-4659-9f64-204eb7c41ed2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.561033 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.561645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.562010 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.562714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.563383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.565076 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.571258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.571258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.571258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.571258 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] PATH_INFO: `/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f/users/d1cbeb06c3ca4ded9e3e060e349be2c8/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.572787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2083ae9a19c4575b2d29b4be9d01f0f, user_id=d1cbeb06c3ca4ded9e3e060e349be2c8, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.576758 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea333-f3f7-4bbc-92ac-42049432c007 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.578696 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1233/2479] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:41.582983 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.597541 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a27f4-0fbb-4487-a983-116d94f4ebc5 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.600245 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1247/2480] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:41 2022] PUT /identity/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f/users/d1cbeb06c3ca4ded9e3e060e349be2c8/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:41.618598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.619102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.619459 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.665287 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] MFA Rules not processed for user `d1cbeb06c3ca4ded9e3e060e349be2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:41.666526 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:41.692111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.692804 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.693090 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.693374 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] PATH_INFO: `/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb/users/e08be9d9d1d745a8932a73ccb58a9b34/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.695429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba2102cf0a14b0fbe3dfe794fbc2bbb, user_id=e08be9d9d1d745a8932a73ccb58a9b34, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.698232 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb508-c153-433f-986c-026d3f33a769 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.712303 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1234/2481] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:41 2022] PUT /identity/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb/users/e08be9d9d1d745a8932a73ccb58a9b34/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:41.723517 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.723852 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.724142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.763166 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] MFA Rules not processed for user `e08be9d9d1d745a8932a73ccb58a9b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:41.763953 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1248/2482] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:41.764886 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:41.776547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.778080 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:41.813253 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:41.846431 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5c70-5352-47a6-8f76-a398842896fc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.850157 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1235/2483] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:41.863072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:41.868231 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:41.942232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.942865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.943151 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.943430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.965283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660816ff-fdd6-4aa7-add0-1674a26ee691 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:41.973573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:41.974738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:41.975332 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:41.975981 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:41.989539 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:41.991251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:41.996040 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:42.003240 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1236/2484] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:42.027187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:42.028724 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:42.040098 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1249/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:42.072262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:42.136120 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c449cab-bfe3-41d0-a7a5-02f44c481b70 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:42.140987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:42.141543 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:42.141831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:42.142111 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] PATH_INFO: `/v3/users/c2329d57275d4f1a999b2d05e0e36ec1` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:42.143256 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2329d57275d4f1a999b2d05e0e36ec1)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:42.145335 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:42.286912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:42.296027 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508227-7ffc-4ca2-be02-7ea1bf0ad8bd service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:42.300656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:42.301330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:42.301598 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:42.301879 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:42.364715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2329d57275d4f1a999b2d05e0e36ec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:42.365344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2329d57275d4f1a999b2d05e0e36ec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:42.365472 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:42.367096 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeef87-0ddb-45ba-879d-eb55ade81bf0 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:42.377700 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1250/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:42.383641 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2329d57275d4f1a999b2d05e0e36ec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:42.465987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2329d57275d4f1a999b2d05e0e36ec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:42.474561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2329d57275d4f1a999b2d05e0e36ec1'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:42.481812 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1237/2487] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:42 2022] DELETE /identity/v3/users/c2329d57275d4f1a999b2d05e0e36ec1 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:42.486785 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:42.487078 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:42.487360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:42.508727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:42.510066 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:42.537705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:42.572540 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1251/2488] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:42.583062 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:42.584515 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:42.644741 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9272d03-bfe0-4d95-9069-711aa065f943 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:42.648716 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:42.649340 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:42.649624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:42.649904 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:42.660169 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:42.661650 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:42.672973 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1238/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:43.158534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:43.160556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:43.173663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b14f1-ae00-4fc3-b7aa-37e918493405 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:43.178930 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:43.179908 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:43.180355 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:43.180740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] PATH_INFO: `/v3/projects/0f8fdca8399d4d3b8f7dac71e4db43cc` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:43.182039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8fdca8399d4d3b8f7dac71e4db43cc)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:43.184215 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:43.301704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8fdca8399d4d3b8f7dac71e4db43cc', 'request_id': 'req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.307417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8fdca8399d4d3b8f7dac71e4db43cc', 'request_id': 'req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.369172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8fdca8399d4d3b8f7dac71e4db43cc', 'request_id': 'req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.382915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] The token cache is being invalidate because project 0f8fdca8399d4d3b8f7dac71e4db43cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:43.383164 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.383804 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.385359 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1252/2490] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:43 2022] DELETE /identity/v3/projects/0f8fdca8399d4d3b8f7dac71e4db43cc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:43.389551 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:43.399482 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f2d7-58e4-42d6-8d07-b07bce232e83 service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:43.416528 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:43.450196 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:43.451763 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:43.466182 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:43.466839 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:43.467251 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:43.467534 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] PATH_INFO: `/v3/users/d1cbeb06c3ca4ded9e3e060e349be2c8` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:43.469720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cbeb06c3ca4ded9e3e060e349be2c8)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:43.471868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:43.515284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f15a-a3c2-4cb1-9f5f-6f33bbd658d8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:43.521571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:43.522615 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:43.523112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:43.523617 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:43.526499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:43.547260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:43.551880 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:43.559557 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1253/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:43.650603 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cbeb06c3ca4ded9e3e060e349be2c8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.651779 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cbeb06c3ca4ded9e3e060e349be2c8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.677826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cbeb06c3ca4ded9e3e060e349be2c8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.792430 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cbeb06c3ca4ded9e3e060e349be2c8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.801105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-d102178e-069f-422e-82cc-eb8c1dcd5d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cbeb06c3ca4ded9e3e060e349be2c8'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:43.809052 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1239/2492] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:43 2022] DELETE /identity/v3/users/d1cbeb06c3ca4ded9e3e060e349be2c8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:44.394456 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:44.396521 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:44.410415 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c792c-23a3-42f4-861e-c8afa8740546 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:44.485261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:44.494988 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:44.495672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:44.495957 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:44.496229 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] PATH_INFO: `/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:44.497384 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2083ae9a19c4575b2d29b4be9d01f0f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:44.500437 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:44.649947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2083ae9a19c4575b2d29b4be9d01f0f', 'request_id': 'req-de1b559e-c10c-4098-8da6-0405b9232b2f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:44.657647 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2083ae9a19c4575b2d29b4be9d01f0f', 'request_id': 'req-de1b559e-c10c-4098-8da6-0405b9232b2f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:44.759865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2083ae9a19c4575b2d29b4be9d01f0f', 'request_id': 'req-de1b559e-c10c-4098-8da6-0405b9232b2f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:44.776154 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] The token cache is being invalidate because project f2083ae9a19c4575b2d29b4be9d01f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:44.776720 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:44.777523 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-de1b559e-c10c-4098-8da6-0405b9232b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:44.784972 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1254/2493] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:44 2022] DELETE /identity/v3/projects/f2083ae9a19c4575b2d29b4be9d01f0f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:44.786017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:44.786889 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:44.792549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:44.836366 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:44.845160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:44.928445 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1240/2494] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:44.936704 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:44.937838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:44.938357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:44.961114 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:44.962666 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:44.972529 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1255/2495] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:44.977885 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:44.990053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306822a-afb0-47a0-953d-677914d1cc96 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:44.997387 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:44.998452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:44.998917 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:44.999408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.000916 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:45.006642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.006642 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.017195 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1241/2496] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:20:45.021279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:45.033367 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c88f-91b4-4fc7-88a9-8ee24cef8972 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:45.039947 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:45.040949 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.041445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.041938 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.042915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.045500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.227315 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1256/2497] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:45 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:45.228161 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:45.243473 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db528-91e7-4c83-a95c-6f21dd20f893 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:45.276019 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:45.330359 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:45.331013 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.331297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.331812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.332494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.334615 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.435636 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1242/2498] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:45 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:45.437490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:45.448838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be797-b695-41ed-8eef-58b3966b7b3e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:45.458024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:45.459165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.459794 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.460323 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.461357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.463709 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.480174 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1257/2499] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:45.482969 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:45.495691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e54e6-6b64-4663-8961-27c9eba0f51a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:45.501097 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:45.501850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.502155 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.502429 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.503105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.504865 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6d06-122a-431b-b20d-d61159d60c92 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.519631 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1243/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:45.524503 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:45.539152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425df4b9-2ddc-47eb-a674-2b334fd5bd20 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:45.555872 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:45.557219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.557500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.557780 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] PATH_INFO: `/v3/projects/070c1051940e486fb8791cb51449f261/users/0fcebcf6f45d4ba7bceef18c992dc1f8/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.584865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=070c1051940e486fb8791cb51449f261, user_id=0fcebcf6f45d4ba7bceef18c992dc1f8, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:45.593146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:45.692486 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1258/2501] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:45 2022] PUT /identity/v3/projects/070c1051940e486fb8791cb51449f261/users/0fcebcf6f45d4ba7bceef18c992dc1f8/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:45.713452 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:45.713961 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:45.714934 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:45.772255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] MFA Rules not processed for user `0fcebcf6f45d4ba7bceef18c992dc1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:45.774343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:45.936553 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1244/2502] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:45.943330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.018644 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f7c8-ee68-4255-98e1-e2aa7fdc58dd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.023538 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.024179 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.024471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.024754 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.025417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.027372 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.097801 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1259/2503] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:45 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:46.102856 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.118646 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ff37-0127-4e5e-beb9-8114f0ab3c52 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.186297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.186297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.186631 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.186908 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.188487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.190300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.301443 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1245/2504] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:46 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:46.302345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.313023 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b423bdb-a5d4-462c-afd6-dd92e537a345 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.317591 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.318224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.318502 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.318775 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.319417 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.321041 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.323398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:46.332771 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1260/2505] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:46.337123 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.347962 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba2e0-c348-49f9-9588-a31356d8c486 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.353168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.353786 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.354065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.354338 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.355018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.356659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968da70-ab8d-4691-9b03-30c97bcee3ec admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.371781 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1246/2506] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:46.373217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.384865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623f646-fdd0-4c3d-bbd2-ebbb2cb58d9d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.389975 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.390820 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.391112 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.391385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] PATH_INFO: `/v3/projects/be6ffd9a594e41d3a7401acd2a77d656/users/a8c6306146194c2db27cf811bc351441/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.408560 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6ffd9a594e41d3a7401acd2a77d656, user_id=a8c6306146194c2db27cf811bc351441, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.411269 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.501456 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1261/2507] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:46 2022] PUT /identity/v3/projects/be6ffd9a594e41d3a7401acd2a77d656/users/a8c6306146194c2db27cf811bc351441/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:46.511926 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.512290 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.512605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.540617 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:46.543538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] MFA Rules not processed for user `a8c6306146194c2db27cf811bc351441`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:46.544801 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:46.618749 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1247/2508] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:46.623303 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.715140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcb2b-65ff-4b37-ad8b-855911ececdf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.721386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.722111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.722471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.722840 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.723609 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.725901 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:46.860310 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1262/2509] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:46 2022] POST /identity/v3/projects => generated 379 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:46.865042 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:46.886866 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9105b2-9096-49b0-9077-6a0b8af5a717 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:46.962107 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:46.962820 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:46.963025 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:46.963297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:46.964012 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:46.966055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.078020 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1248/2510] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:46 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:47.081614 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.093044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14c0a-b5e2-4fcf-bae8-3753a7e39516 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.100016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.100728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.101132 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.101413 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.102074 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.104044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.121414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.121880 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1263/2511] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:47.132952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a0bde-f58e-44d9-9bc0-632f8159911e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.138108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.139142 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.139424 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.139872 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] PATH_INFO: `/v3/projects/fa3dacd9c48146c8b57599f491bac845/users/84bd1cc6c9404db9a92e0f410bf21ac6/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.157596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3dacd9c48146c8b57599f491bac845, user_id=84bd1cc6c9404db9a92e0f410bf21ac6, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.160357 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.286152 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1249/2512] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:47 2022] PUT /identity/v3/projects/fa3dacd9c48146c8b57599f491bac845/users/84bd1cc6c9404db9a92e0f410bf21ac6/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:47.290726 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.301999 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea3446-21df-4965-a5c8-3cf7ba407add admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.361108 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.361725 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.362008 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.362275 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.362912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.364522 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027da6c-8031-40e5-8338-0f805fdcd37a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.374299 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1264/2513] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:47.379143 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.401537 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd291-ebe0-4f51-a290-e24aef39d960 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.407408 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.408352 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.408709 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.409065 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] PATH_INFO: `/v3/projects/fa3dacd9c48146c8b57599f491bac845/users/84bd1cc6c9404db9a92e0f410bf21ac6/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.411150 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3dacd9c48146c8b57599f491bac845, user_id=84bd1cc6c9404db9a92e0f410bf21ac6, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.414071 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.426976 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1250/2514] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:47 2022] PUT /identity/v3/projects/fa3dacd9c48146c8b57599f491bac845/users/84bd1cc6c9404db9a92e0f410bf21ac6/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:47.438642 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.439388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.439388 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.473476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] MFA Rules not processed for user `84bd1cc6c9404db9a92e0f410bf21ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:47.474708 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:47.512475 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:47.567476 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1265/2515] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:47.582053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.584129 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:47.661121 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf547-1ddd-430a-8829-3fb294bbacee admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.666039 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.666390 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.666711 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.666949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.677720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.679215 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.683417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:47.689803 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1251/2516] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:47.815808 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:47.825656 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc0328-4e2b-425f-b578-5ca33df34606 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:47.829876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:47.836715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:47.836715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:47.836715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:47.910466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:47.912024 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:47.922862 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1266/2517] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:51.506156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:51.507673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:51.554556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:51.556088 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:51.587043 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a0998-26dd-49b8-b63c-7d8a2fa13586 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:51.591722 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:51.592373 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:51.592833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:51.593124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] PATH_INFO: `/v3/users/94c64f76058e493d958ec3f96a56a0e0` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:51.594297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c64f76058e493d958ec3f96a56a0e0)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:51.596381 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:51.635677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1732f-5ed3-401e-ac8c-68de91566a7f service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:51.640310 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:51.640959 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:51.641240 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:51.641517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] PATH_INFO: `/v3/users/4af60a16a7c24542b5b8aa94d2bfe9ef` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:51.642678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af60a16a7c24542b5b8aa94d2bfe9ef)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:51.644660 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:51.795755 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c64f76058e493d958ec3f96a56a0e0'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.797291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c64f76058e493d958ec3f96a56a0e0'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.810156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c64f76058e493d958ec3f96a56a0e0'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.875009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af60a16a7c24542b5b8aa94d2bfe9ef'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.875738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af60a16a7c24542b5b8aa94d2bfe9ef'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.882798 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af60a16a7c24542b5b8aa94d2bfe9ef'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.883450 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af60a16a7c24542b5b8aa94d2bfe9ef'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.894140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:51.896977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af60a16a7c24542b5b8aa94d2bfe9ef'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.900821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c64f76058e493d958ec3f96a56a0e0'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.910072 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-26d925c1-b7ee-4ac5-ae53-67481cf90f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c64f76058e493d958ec3f96a56a0e0'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:51.916348 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1252/2518] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:51 2022] DELETE /identity/v3/users/94c64f76058e493d958ec3f96a56a0e0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:51.921561 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:51.922298 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:51.922580 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:51.942088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:51.943313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:51.987307 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1267/2519] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:51 2022] DELETE /identity/v3/users/4af60a16a7c24542b5b8aa94d2bfe9ef => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:52.010049 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1253/2520] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:52.020738 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:52.022205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:52.088634 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97434d-25bf-4c5f-872a-057e325c671e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:52.092499 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:52.093130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:52.093909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:52.094184 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:52.162064 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:52.163513 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:52.174921 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1268/2521] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:52.274789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:52.279690 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:52.309220 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1770ee-cc08-4544-ad0d-d774fe0cd583 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:52.317046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:52.318159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:52.318674 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:52.319188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] PATH_INFO: `/v3/projects/55f0064a8bc14f30b2749047dfd2f53d` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:52.321266 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f0064a8bc14f30b2749047dfd2f53d)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:52.325356 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:52.347344 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:52.417745 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:52.419529 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:52.432541 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37abd73-1997-4f69-8c90-20b1e5035d16 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:52.443334 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:52.444077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:52.444356 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:52.444628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] PATH_INFO: `/v3/users/0fcebcf6f45d4ba7bceef18c992dc1f8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:52.446007 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcebcf6f45d4ba7bceef18c992dc1f8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:52.448270 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:52.466497 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f0064a8bc14f30b2749047dfd2f53d', 'request_id': 'req-bd90f01d-51e7-460c-ba75-606acf547089'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.476189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f0064a8bc14f30b2749047dfd2f53d', 'request_id': 'req-bd90f01d-51e7-460c-ba75-606acf547089'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.488069 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f0064a8bc14f30b2749047dfd2f53d', 'request_id': 'req-bd90f01d-51e7-460c-ba75-606acf547089'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.717748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] The token cache is being invalidate because project 55f0064a8bc14f30b2749047dfd2f53d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:52.717748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.717748 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.717748 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1254/2522] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:52 2022] DELETE /identity/v3/projects/55f0064a8bc14f30b2749047dfd2f53d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:52.718645 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:52.721066 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:52.801241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f01d-51e7-460c-ba75-606acf547089 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:52.801895 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcebcf6f45d4ba7bceef18c992dc1f8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.802641 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcebcf6f45d4ba7bceef18c992dc1f8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.806687 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:52.807300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:52.807665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:52.807892 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] PATH_INFO: `/v3/projects/bbd9de5d7b7d4b518c27122aec803dba` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:52.809108 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd9de5d7b7d4b518c27122aec803dba)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:52.811279 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcebcf6f45d4ba7bceef18c992dc1f8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.811425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:52.811960 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcebcf6f45d4ba7bceef18c992dc1f8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.836493 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcebcf6f45d4ba7bceef18c992dc1f8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.919316 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1269/2523] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:52 2022] DELETE /identity/v3/users/0fcebcf6f45d4ba7bceef18c992dc1f8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:52.924755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:52.924755 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:52.937835 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:52.940425 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd9de5d7b7d4b518c27122aec803dba', 'request_id': 'req-0bd9a6aa-000e-46f3-82c2-e6c016349958'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.947283 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd9de5d7b7d4b518c27122aec803dba', 'request_id': 'req-0bd9a6aa-000e-46f3-82c2-e6c016349958'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.957855 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd9de5d7b7d4b518c27122aec803dba', 'request_id': 'req-0bd9a6aa-000e-46f3-82c2-e6c016349958'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:52.999242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a279567-f82c-49bd-9e9e-aa93974d49b7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:53.004682 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:53.005329 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:53.005612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:53.005889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] PATH_INFO: `/v3/users/c5cb5f5e3fdb46169809497665ef4875` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:53.007810 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cb5f5e3fdb46169809497665ef4875)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:53.011796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:53.038828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] The token cache is being invalidate because project bbd9de5d7b7d4b518c27122aec803dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:53.039125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.040382 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-0bd9a6aa-000e-46f3-82c2-e6c016349958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.041460 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1255/2524] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:52 2022] DELETE /identity/v3/projects/bbd9de5d7b7d4b518c27122aec803dba => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:53.042721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:53.043007 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:53.043287 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:53.073929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:53.073929 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:53.073929 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1256/2525] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:53.074438 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:53.075717 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:53.088628 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d66b-486b-44c2-9b68-236a35f7dccc None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:53.093744 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:53.094395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:53.094675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:53.096984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] PATH_INFO: `/v3/users/e08be9d9d1d745a8932a73ccb58a9b34` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:53.096984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08be9d9d1d745a8932a73ccb58a9b34)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:53.098370 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:53.122913 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:53.326386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cb5f5e3fdb46169809497665ef4875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.326998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cb5f5e3fdb46169809497665ef4875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.335362 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cb5f5e3fdb46169809497665ef4875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.336140 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cb5f5e3fdb46169809497665ef4875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.355906 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cb5f5e3fdb46169809497665ef4875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.381264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08be9d9d1d745a8932a73ccb58a9b34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.381939 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08be9d9d1d745a8932a73ccb58a9b34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.393987 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08be9d9d1d745a8932a73ccb58a9b34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.466133 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1270/2526] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:52 2022] DELETE /identity/v3/users/c5cb5f5e3fdb46169809497665ef4875 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:53.467216 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:53.470629 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:53.523941 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08be9d9d1d745a8932a73ccb58a9b34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.534718 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08be9d9d1d745a8932a73ccb58a9b34'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:53.535284 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bb8a5-5ca7-40b1-a7d7-a510087c52d8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:53.541059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:53.541742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:53.542031 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:53.542353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:53.543058 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1257/2527] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:53 2022] DELETE /identity/v3/users/e08be9d9d1d745a8932a73ccb58a9b34 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:53.545909 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:53.617786 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:53.644453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:53.646548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:53.660500 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1271/2528] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:54.050715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.052311 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.125869 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c256b7-f58b-4eb4-b2c9-de6c1f11d297 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.130768 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.131272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.131556 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.131896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] PATH_INFO: `/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.132980 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8256fc8c274c61a65e34d10a6bf30f)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.134957 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:54.136876 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.138479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.151650 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076684f-ca4c-499c-bf0c-cbd47ced4069 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.159347 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.160195 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.160478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.160937 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] PATH_INFO: `/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.164387 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba2102cf0a14b0fbe3dfe794fbc2bbb)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.167672 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:54.350636 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8256fc8c274c61a65e34d10a6bf30f', 'request_id': 'req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.350897 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba2102cf0a14b0fbe3dfe794fbc2bbb', 'request_id': 'req-77bd9009-92c7-4ca7-8e81-f1095114c503'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.357309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8256fc8c274c61a65e34d10a6bf30f', 'request_id': 'req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.358476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba2102cf0a14b0fbe3dfe794fbc2bbb', 'request_id': 'req-77bd9009-92c7-4ca7-8e81-f1095114c503'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.365011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8256fc8c274c61a65e34d10a6bf30f', 'request_id': 'req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.534865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba2102cf0a14b0fbe3dfe794fbc2bbb', 'request_id': 'req-77bd9009-92c7-4ca7-8e81-f1095114c503'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.547396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] The token cache is being invalidate because project 1ba2102cf0a14b0fbe3dfe794fbc2bbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:54.547732 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.548385 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.549887 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1272/2529] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:54 2022] DELETE /identity/v3/projects/1ba2102cf0a14b0fbe3dfe794fbc2bbb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:54.550711 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.552094 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.618838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd9009-92c7-4ca7-8e81-f1095114c503 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.653967 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] The token cache is being invalidate because project 6a8256fc8c274c61a65e34d10a6bf30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:54.654268 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.654990 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-de7fbbf6-eb8d-4d0e-b967-2c46265d71fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.656534 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1258/2530] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:54 2022] DELETE /identity/v3/projects/6a8256fc8c274c61a65e34d10a6bf30f => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:54.657863 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdca5b-5bbe-446b-b9c1-edbe3a1e760c None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.658151 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdca5b-5bbe-446b-b9c1-edbe3a1e760c None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.658423 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdca5b-5bbe-446b-b9c1-edbe3a1e760c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.692655 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.693424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.693715 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.693992 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] PATH_INFO: `/v3/projects/070c1051940e486fb8791cb51449f261` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.695301 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=070c1051940e486fb8791cb51449f261)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.697378 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:54.704781 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-b6fdca5b-5bbe-446b-b9c1-edbe3a1e760c None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:54.706862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-b6fdca5b-5bbe-446b-b9c1-edbe3a1e760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.710446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:54.727910 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1259/2531] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:54.789789 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.790115 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.790476 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.812292 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:54.813838 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.821900 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1260/2532] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:54.826255 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.836294 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c871e0-5d1d-4375-83cd-bcb27f10d5f2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.841624 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.842264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.842549 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.842821 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.844607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:54.844607 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.846203 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:54.856237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:54.864444 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '070c1051940e486fb8791cb51449f261', 'request_id': 'req-c232dbe1-ed07-4a44-a1b0-d575801fa093'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.864701 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1261/2533] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:20:54.865940 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.870705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '070c1051940e486fb8791cb51449f261', 'request_id': 'req-c232dbe1-ed07-4a44-a1b0-d575801fa093'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.878233 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d64d8-a3a7-4699-a6ee-cf38cf68f4b1 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.942385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.943020 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.943300 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.943573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.944270 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.946285 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:54.952317 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '070c1051940e486fb8791cb51449f261', 'request_id': 'req-c232dbe1-ed07-4a44-a1b0-d575801fa093'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.966613 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] The token cache is being invalidate because project 070c1051940e486fb8791cb51449f261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:54.966905 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.967597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:54.969670 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1273/2534] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:54 2022] DELETE /identity/v3/projects/070c1051940e486fb8791cb51449f261 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:54.973557 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:54.974853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:54.988568 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232dbe1-ed07-4a44-a1b0-d575801fa093 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:54.993389 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:54.999119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:54.999119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:54.999119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] PATH_INFO: `/v3/users/a8c6306146194c2db27cf811bc351441` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:54.999119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c6306146194c2db27cf811bc351441)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:54.999119 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.063288 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1262/2535] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:54 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:55.064222 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.065894 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:55.129264 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967efcc-0f21-43a3-8809-37375e323b49 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.134030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.134633 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.134912 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.135186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.135842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.137850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.227046 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1263/2536] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:55 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:20:55.231732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.241833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206792-9919-4c63-b03d-33a205889f57 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.246596 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.247188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.247474 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.247784 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.248489 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.250087 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.264419 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1264/2537] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:55.267997 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.272014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c6306146194c2db27cf811bc351441'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:55.273037 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c6306146194c2db27cf811bc351441'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:55.278595 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dbef3-0730-44e4-85d7-371eae6ff98e admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.281089 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c6306146194c2db27cf811bc351441'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:55.282338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c6306146194c2db27cf811bc351441'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:55.284386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.285006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.285291 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.285613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.286414 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.288100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.296500 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2e31a685-de66-41d3-b28a-101893ab6d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c6306146194c2db27cf811bc351441'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:55.300105 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1265/2538] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:55.304426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.306154 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:55.319116 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0697d4-f20f-4f8e-929a-0a3e0e3c442a admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.323811 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.324485 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.324829 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.325050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] PATH_INFO: `/v3/projects/d7c45fbae2a640aebb1837f6d372ca1c/users/f6b6b887610b4b43a3952353abb715a5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.341484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c45fbae2a640aebb1837f6d372ca1c, user_id=f6b6b887610b4b43a3952353abb715a5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.345105 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.375227 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1274/2539] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:54 2022] DELETE /identity/v3/users/a8c6306146194c2db27cf811bc351441 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:55.461546 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1266/2540] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:55 2022] PUT /identity/v3/projects/d7c45fbae2a640aebb1837f6d372ca1c/users/f6b6b887610b4b43a3952353abb715a5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:55.472203 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.472492 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.472770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.501466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] MFA Rules not processed for user `f6b6b887610b4b43a3952353abb715a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:55.502680 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:55.565495 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:55.569599 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1275/2541] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:55.574453 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.613905 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:55.643372 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732f5c-aa9c-47a1-bdef-9135cf86a83d admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.649037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.649659 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.649945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.650223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.650878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.653004 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.750564 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1267/2542] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:55 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:55.752871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.763593 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233bdff-e247-4ef3-b449-14c543bf6623 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.822612 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.823142 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.823428 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.823741 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.824396 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.826357 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.840599 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.842574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:55.858754 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196fe26-098a-4f8d-a7dd-3775e7038cba admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.864125 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.864915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.865050 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.865358 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] PATH_INFO: `/v3/projects/be6ffd9a594e41d3a7401acd2a77d656` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.866663 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6ffd9a594e41d3a7401acd2a77d656)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.868621 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.929868 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1276/2543] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:55 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:55.933883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.944806 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068af38-618d-4501-94dc-6a680f234701 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.949454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.950075 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.950353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.950627 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:55.951261 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:55.952886 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:55.963046 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1277/2544] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:55.967346 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:55.977916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441aa1f-fe2c-44f3-847c-ed9dbc41a9ff admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:55.982797 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:55.983445 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:55.983765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:55.984044 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] PATH_INFO: `/v3/projects/24ff324ca91a4357b1fdcf092be8da39/users/3e7fee8559744968a4efb3d3a8928e72/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.002120 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ff324ca91a4357b1fdcf092be8da39, user_id=3e7fee8559744968a4efb3d3a8928e72, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.005021 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.005788 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6ffd9a594e41d3a7401acd2a77d656', 'request_id': 'req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.011570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6ffd9a594e41d3a7401acd2a77d656', 'request_id': 'req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.018034 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6ffd9a594e41d3a7401acd2a77d656', 'request_id': 'req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.071946 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1278/2545] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:55 2022] PUT /identity/v3/projects/24ff324ca91a4357b1fdcf092be8da39/users/3e7fee8559744968a4efb3d3a8928e72/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:56.076047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:56.100306 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] The token cache is being invalidate because project be6ffd9a594e41d3a7401acd2a77d656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:56.100920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.101383 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.103451 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1268/2546] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:55 2022] DELETE /identity/v3/projects/be6ffd9a594e41d3a7401acd2a77d656 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:56.116574 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:56.117945 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:56.142912 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9411-74d3-42ef-b5e9-bbbca3863622 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:56.148714 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:56.152701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.152701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.152701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.152701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.152701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.163884 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1279/2547] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:20:56.168130 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:56.169859 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:56.183471 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97ae0b-d31a-4f91-8c25-b7a18cfe68f1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:56.184274 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbed14-1ff5-454e-9341-0cd6f4e330e7 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:56.189205 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:56.189858 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.190153 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.190411 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] PATH_INFO: `/v3/projects/24ff324ca91a4357b1fdcf092be8da39/users/3e7fee8559744968a4efb3d3a8928e72/roles/59156af11c8d4db5aa049e4462ed2ed0` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.190544 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:56.191186 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.191639 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.191935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] PATH_INFO: `/v3/users/84bd1cc6c9404db9a92e0f410bf21ac6` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.192479 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ff324ca91a4357b1fdcf092be8da39, user_id=3e7fee8559744968a4efb3d3a8928e72, role_id=59156af11c8d4db5aa049e4462ed2ed0)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.193149 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84bd1cc6c9404db9a92e0f410bf21ac6)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.195189 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.195339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a049b1a-1be4-4124-b578-33086f027829 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.205932 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1280/2548] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:56 2022] PUT /identity/v3/projects/24ff324ca91a4357b1fdcf092be8da39/users/3e7fee8559744968a4efb3d3a8928e72/roles/59156af11c8d4db5aa049e4462ed2ed0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:56.217039 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.217350 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.217631 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.247466 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] MFA Rules not processed for user `3e7fee8559744968a4efb3d3a8928e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:56.255744 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:56.274146 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:56.316918 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1281/2549] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:20:56.327964 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:56.329325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:56.386067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144c3c7-a955-4eb3-8b41-03024ace9e70 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:56.389770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:56.390401 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.390689 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.390967 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.400231 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.401740 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.412090 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1282/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:20:56.495428 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:56.501385 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84bd1cc6c9404db9a92e0f410bf21ac6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.502044 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84bd1cc6c9404db9a92e0f410bf21ac6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.513812 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84bd1cc6c9404db9a92e0f410bf21ac6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.601455 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84bd1cc6c9404db9a92e0f410bf21ac6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.610172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84bd1cc6c9404db9a92e0f410bf21ac6'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:56.616767 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1269/2551] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:20:56 2022] DELETE /identity/v3/users/84bd1cc6c9404db9a92e0f410bf21ac6 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:20:56.692596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:56.702451 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c62b1d-aba9-45dc-8991-39bc49a48d0b service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:56.764883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:56.765505 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:56.766328 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:56.766606 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:56.768449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:56.835572 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:56.837159 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8552838-6438-4642-bf62-8de9d43e1d0d service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:56.849452 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1283/2552] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:20:57.072763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.074254 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:57.148653 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1720b-9a62-43c1-806f-b756a4eb6d01 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.153091 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.153720 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.154006 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.154281 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] PATH_INFO: `/v3/projects/fa3dacd9c48146c8b57599f491bac845` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.155395 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3dacd9c48146c8b57599f491bac845)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.157867 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.295223 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3dacd9c48146c8b57599f491bac845', 'request_id': 'req-8144e42e-eae7-4170-b672-01f6afad237e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:57.301141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3dacd9c48146c8b57599f491bac845', 'request_id': 'req-8144e42e-eae7-4170-b672-01f6afad237e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:57.307874 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3dacd9c48146c8b57599f491bac845', 'request_id': 'req-8144e42e-eae7-4170-b672-01f6afad237e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:57.424313 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] The token cache is being invalidate because project fa3dacd9c48146c8b57599f491bac845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:20:57.424540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:57.425193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-8144e42e-eae7-4170-b672-01f6afad237e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:20:57.426791 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1270/2553] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:20:57 2022] DELETE /identity/v3/projects/fa3dacd9c48146c8b57599f491bac845 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:57.433435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.433435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.433913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.457427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:57.458936 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:57.477016 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:57.523118 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1284/2554] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:57.531669 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.531958 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.532237 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.552807 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:57.554014 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:57.561917 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1271/2555] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:57.566748 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.576705 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2fa9a-070c-44ec-b8cf-3fa41ea70aaa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.581871 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.582543 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.582828 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.583104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.583864 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:20:57.584290 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.585965 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.597967 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1285/2556] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:20:57.602877 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.613678 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e5d3-c077-48fd-b941-20fbe4043a70 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.618535 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.629949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.629949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.629949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.629949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.629949 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.726022 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1272/2557] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:20:57 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:57.730277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.741144 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d614a11-836c-4e16-9077-b9686b22fc3f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.798747 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.799353 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.799679 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.799958 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.800594 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.802531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.912199 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1286/2558] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:20:57 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:20:57.916031 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.926514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b6b-1c44-48a9-8bae-44a39833b6e5 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.931221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.931862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.932202 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.932484 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.933130 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.934732 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.945257 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1273/2559] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:57.949339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.960131 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcfebe-1052-4418-9d20-85c1e77a22b8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.964795 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.965391 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.965677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.965956 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:57.966607 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:57.968242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d0c4f-db41-40f1-b9c0-4d5dd5184b16 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:57.979359 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1287/2560] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:20:57.983713 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:57.993635 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0839-0456-4974-b905-a2bb74b0b039 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:57.998221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:57.998858 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:57.999141 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:57.999411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] PATH_INFO: `/v3/projects/5fc5163150b440ebb6e60097378e2274/users/b62335db37664c68802042ff7c22fc33/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:58.002188 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:20:58.014978 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc5163150b440ebb6e60097378e2274, user_id=b62335db37664c68802042ff7c22fc33, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:58.018510 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:58.117965 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1274/2561] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:20:57 2022] PUT /identity/v3/projects/5fc5163150b440ebb6e60097378e2274/users/b62335db37664c68802042ff7c22fc33/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:20:58.130710 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b744d03-bd4d-4853-a8f8-2f7d8903d675 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:58.130998 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b744d03-bd4d-4853-a8f8-2f7d8903d675 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:58.131278 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b744d03-bd4d-4853-a8f8-2f7d8903d675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:58.161866 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6b744d03-bd4d-4853-a8f8-2f7d8903d675 None None] MFA Rules not processed for user `b62335db37664c68802042ff7c22fc33`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:20:58.163104 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6b744d03-bd4d-4853-a8f8-2f7d8903d675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:58.229750 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1288/2562] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:20:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:20:58.241471 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:20:58.243295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:20:58.303950 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69d6d-3ce5-41d4-a36f-601e3f9ce977 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:20:58.307714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:20:58.308333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:20:58.308613 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:20:58.308895 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:20:58.317819 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:20:58.319262 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:20:58.328243 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1275/2563] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:20:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:00.158467 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:00.159000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:00.159000 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:00.180139 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] MFA Rules not processed for user `b62335db37664c68802042ff7c22fc33`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:00.181370 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:00.188507 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1289/2564] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:00.926307 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:00.927851 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:00.987067 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:00.988490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:01.000111 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:01.004011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec999f-2701-4b18-820f-395fed2bf415 service cinder] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:01.009759 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:01.010417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:01.010710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:01.010984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] PATH_INFO: `/v3/users/f6b6b887610b4b43a3952353abb715a5` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:01.012194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b6b887610b4b43a3952353abb715a5)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:01.014331 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:01.065846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa1be-1a78-4a23-847b-fcdb89626b62 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:01.070946 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:01.071553 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:01.071877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:01.072148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:01.073878 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:01.085583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:01.087165 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d91f-7aa1-41cb-b236-72cf21238f17 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:01.097166 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1290/2565] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:01.202030 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b6b887610b4b43a3952353abb715a5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:01.202728 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b6b887610b4b43a3952353abb715a5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:01.216738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b6b887610b4b43a3952353abb715a5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:01.305984 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b6b887610b4b43a3952353abb715a5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:01.314915 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b6b887610b4b43a3952353abb715a5'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:01.321630 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1276/2566] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:00 2022] DELETE /identity/v3/users/f6b6b887610b4b43a3952353abb715a5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:01.326753 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:01.327518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:01.327518 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:01.346951 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:01.348206 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:01.407901 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1291/2567] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:01.418318 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:01.419772 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:01.426898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:01.479260 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976b8ae-1e09-459f-92ac-40360cfac8a2 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:01.483076 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:01.483735 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:01.484017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:01.484322 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:01.495548 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:01.497018 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:01.506951 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1277/2568] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:01.986204 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:01.987943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:02.000102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee74d2-fa09-4bcf-b2a5-0d3cf6576e2a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:02.009141 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:02.010176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:02.010463 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:02.010742 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] PATH_INFO: `/v3/projects/d7c45fbae2a640aebb1837f6d372ca1c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:02.012006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c45fbae2a640aebb1837f6d372ca1c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:02.025014 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:02.153610 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c45fbae2a640aebb1837f6d372ca1c', 'request_id': 'req-d438ae58-bff6-4f57-a430-1fc0b3c5efea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.160321 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c45fbae2a640aebb1837f6d372ca1c', 'request_id': 'req-d438ae58-bff6-4f57-a430-1fc0b3c5efea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.243514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c45fbae2a640aebb1837f6d372ca1c', 'request_id': 'req-d438ae58-bff6-4f57-a430-1fc0b3c5efea'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.256158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] The token cache is being invalidate because project d7c45fbae2a640aebb1837f6d372ca1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:02.256442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.257117 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.258591 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1292/2569] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:01 2022] DELETE /identity/v3/projects/d7c45fbae2a640aebb1837f6d372ca1c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:02.262763 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:02.273330 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce488-08a1-401e-a28d-57022a5b372e service neutron] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:02.330511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:02.331100 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:02.331379 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:02.331685 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] PATH_INFO: `/v3/users/3e7fee8559744968a4efb3d3a8928e72` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:02.332836 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7fee8559744968a4efb3d3a8928e72)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:02.334734 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:02.536178 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7fee8559744968a4efb3d3a8928e72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.536816 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7fee8559744968a4efb3d3a8928e72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.549878 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7fee8559744968a4efb3d3a8928e72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.641446 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7fee8559744968a4efb3d3a8928e72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.649675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-524929a4-d1e9-4dbf-abe4-63152f2943a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7fee8559744968a4efb3d3a8928e72'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:02.655771 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1278/2570] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:02 2022] DELETE /identity/v3/users/3e7fee8559744968a4efb3d3a8928e72 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:03.136057 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:03.137630 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:03.153070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438ae58-bff6-4f57-a430-1fc0b3c5efea admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:03.179242 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:03.222226 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:03.222943 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.223235 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.223517 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] PATH_INFO: `/v3/projects/24ff324ca91a4357b1fdcf092be8da39` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.226977 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ff324ca91a4357b1fdcf092be8da39)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:03.229339 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:03.381922 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ff324ca91a4357b1fdcf092be8da39', 'request_id': 'req-1069c6c2-d299-40d7-a2d7-6213a3dd9804'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:03.387360 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ff324ca91a4357b1fdcf092be8da39', 'request_id': 'req-1069c6c2-d299-40d7-a2d7-6213a3dd9804'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:03.489427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ff324ca91a4357b1fdcf092be8da39', 'request_id': 'req-1069c6c2-d299-40d7-a2d7-6213a3dd9804'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:03.502618 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] The token cache is being invalidate because project 24ff324ca91a4357b1fdcf092be8da39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:03.503520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:03.504224 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1069c6c2-d299-40d7-a2d7-6213a3dd9804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:03.505742 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1293/2571] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:03 2022] DELETE /identity/v3/projects/24ff324ca91a4357b1fdcf092be8da39 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:03.510815 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.511336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.511336 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.532393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:03.533665 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:03.557279 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:03.606273 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1279/2572] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:03.614916 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.615185 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.615483 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.636639 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:03.637800 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:03.645848 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1294/2573] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:03.650159 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:03.660487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c69b79-8bb4-4727-93bf-5f1be5376ae0 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:03.665888 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:03.666547 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.666873 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.667092 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] PATH_INFO: `/v3/domains` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.668211 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:21:03.668531 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:03.670371 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:03.683543 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1280/2574] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:21:03.688059 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:03.697453 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aa252-5361-44ae-8700-804e8a3a1d73 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:03.702369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:03.702970 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.703250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.703556 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.704232 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:03.706172 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:03.822687 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1295/2575] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:03.826783 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:03.837241 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5502-0852-4793-aca0-1fbf18373160 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:03.904221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:03.904923 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:03.905221 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:03.905504 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] PATH_INFO: `/v3/users` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:03.906164 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:03.908295 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:04.014539 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1281/2576] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:21:03 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:04.015904 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:04.029236 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d836b-4553-49d7-9188-43be4f6054e3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:04.035381 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:04.041476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:04.041476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:04.041476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:04.041476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:04.041476 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:04.055153 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1296/2577] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:04.056666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:04.067082 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a789-9662-4eab-abe2-a90c94c59011 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:04.074239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:04.074239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:04.074239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:04.074239 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:04.074710 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:04.075494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa237ef-0194-4df7-b4f6-4ed5910ccba7 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:04.086801 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1282/2578] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:04.090954 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:04.101520 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64add-266c-4026-95bf-6b45836f0abf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:04.108823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:04.108823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:04.108823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:04.108823 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] PATH_INFO: `/v3/projects/3122b54ae58344d6bfc6c2bb1ae0f981/users/44da1e0d3ed044d28cd897ab77724dd5/roles/670880c95def468fa84d3961a35799aa` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:04.122548 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3122b54ae58344d6bfc6c2bb1ae0f981, user_id=44da1e0d3ed044d28cd897ab77724dd5, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:04.125244 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:04.249305 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1297/2579] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:21:04 2022] PUT /identity/v3/projects/3122b54ae58344d6bfc6c2bb1ae0f981/users/44da1e0d3ed044d28cd897ab77724dd5/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:04.259842 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:04.260136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:04.260416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:04.292862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] MFA Rules not processed for user `44da1e0d3ed044d28cd897ab77724dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:04.292862 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:04.369718 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1283/2580] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:04.375547 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:04.377338 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:04.433592 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:04.464009 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce708b5-47bd-457c-9103-07bb16aa93a0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:04.468288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1b9f055f6c4def891834a8114b04b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:04.468913 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:04.469198 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:04.469478 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:04.480784 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:04.482345 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc1d9-9a78-45de-8714-7779dcde5758 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:04.492441 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1298/2581] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:12.692289 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:12.694017 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:12.723676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:12.771868 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400ba6-58dc-429e-8ed9-a774f1d0275a None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:12.778168 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd66576ed2425a8ad1fe0b69a4c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:12.778826 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:12.779109 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:12.779386 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:12.781213 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:12.794449 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:12.796156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:12.807962 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1284/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:14.677596 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=13 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:19.294796 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895add25-dbab-4978-8cbb-7b43eb287569 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:19.295380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895add25-dbab-4978-8cbb-7b43eb287569 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:19.295380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895add25-dbab-4978-8cbb-7b43eb287569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:19.347544 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-895add25-dbab-4978-8cbb-7b43eb287569 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:19.348831 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-895add25-dbab-4978-8cbb-7b43eb287569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:19.413230 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1299/2583] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:21:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:19.566871 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:19.568398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:19.660540 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078f65d-bd8c-4b1c-97ae-6ebfce15afc1 service nova] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:19.666350 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:19.666985 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:19.667479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:19.667640 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:19.679672 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:19.681814 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:19.691263 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1285/2584] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:21:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:21:20.564187 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408fb2e-6f23-4515-a66c-5ceae06d8822 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:20.564731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408fb2e-6f23-4515-a66c-5ceae06d8822 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:20.564731 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408fb2e-6f23-4515-a66c-5ceae06d8822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:20.584926 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a408fb2e-6f23-4515-a66c-5ceae06d8822 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:20.586158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a408fb2e-6f23-4515-a66c-5ceae06d8822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:20.594074 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1300/2585] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:21:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:20.607227 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:20.619064 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad23df-55aa-47e0-b5db-2d9feabf4a96 service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:20.624806 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:20.625421 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:20.625729 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:20.626027 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:20.636469 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:20.638244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:20.648794 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1286/2586] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:21:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:21:21.385314 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e50b-36bf-45aa-9a7e-5d0314ee4803 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:21.385597 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e50b-36bf-45aa-9a7e-5d0314ee4803 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:21.385874 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e50b-36bf-45aa-9a7e-5d0314ee4803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:21.405843 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6446e50b-36bf-45aa-9a7e-5d0314ee4803 None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:21.407070 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6446e50b-36bf-45aa-9a7e-5d0314ee4803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:21.414302 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1301/2587] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Oct 30 01:21:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:21:21.427326 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:21.438333 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723009fc-10e6-4345-9501-cb46863ab685 service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:21.443879 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:21.444496 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:21.444777 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:21.445055 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:21.454393 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:21.456052 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:21.466723 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1287/2588] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:21:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:21:22.158335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:22.160319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:22.160319 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:22.179649 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] MFA Rules not processed for user `27db563707034f9aa2e8df84704ff249`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:22.180942 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:22.188426 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1302/2589] 10.222.0.26 () {58 vars in 967 bytes} [Sun Oct 30 01:21:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:22.201719 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:22.213367 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d8b52-7bb0-4a1a-a2ee-382becafd176 service swift] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:22.218491 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6a6265c254040d9a0d8ae45b29d3073', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:22.219112 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:22.219398 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:22.219714 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:22.229880 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:22.231516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea88d0-6fa4-4f6a-99e5-e4ebf871473c service swift] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:22.241082 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1288/2590] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Oct 30 01:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:21:25.141207 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:25.142807 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:25.210028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a054dc-c4af-4caf-8b35-8fffa40310fb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:25.215101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:25.215769 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:25.216102 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:25.216335 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] PATH_INFO: `/v3/users/b62335db37664c68802042ff7c22fc33` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:25.217490 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b62335db37664c68802042ff7c22fc33)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:25.219419 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:25.441448 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b62335db37664c68802042ff7c22fc33'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:25.442101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b62335db37664c68802042ff7c22fc33'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:25.448528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:25.450663 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b62335db37664c68802042ff7c22fc33'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:25.451408 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b62335db37664c68802042ff7c22fc33'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:25.464982 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b62335db37664c68802042ff7c22fc33'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:25.579857 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1303/2591] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:25 2022] DELETE /identity/v3/users/b62335db37664c68802042ff7c22fc33 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:25.584404 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:25.584896 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:25.585037 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:25.606411 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:25.607692 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:25.641648 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=100 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:25.669524 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1289/2592] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:25.681398 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:25.688981 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:25.747944 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae7cb-dd71-4e85-8beb-08019e219f54 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:25.751721 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:25.757586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:25.757586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:25.757586 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:25.763176 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:25.764646 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:25.774437 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1304/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:26.084252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:26.085952 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:26.098825 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337f2b-2d99-4f4c-9880-9d8df317a560 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:26.104416 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:26.105058 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:26.105337 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:26.105614 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] PATH_INFO: `/v3/projects/5fc5163150b440ebb6e60097378e2274` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:26.106738 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc5163150b440ebb6e60097378e2274)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:26.108697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:26.307309 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc5163150b440ebb6e60097378e2274', 'request_id': 'req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:26.312946 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc5163150b440ebb6e60097378e2274', 'request_id': 'req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:26.319634 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc5163150b440ebb6e60097378e2274', 'request_id': 'req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:26.464417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] The token cache is being invalidate because project 5fc5163150b440ebb6e60097378e2274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:26.464715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:26.465396 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-a4706c93-c9dd-45b3-9cc6-26ee61aa7ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:26.466938 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1290/2594] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:26 2022] DELETE /identity/v3/projects/5fc5163150b440ebb6e60097378e2274 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:29.736255 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:29.737735 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:29.752250 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dae19-eab3-48b5-9af0-163961a95f87 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:29.818282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:29.818933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:29.819219 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:29.819498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] PATH_INFO: `/v3/users/44da1e0d3ed044d28cd897ab77724dd5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:29.820706 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44da1e0d3ed044d28cd897ab77724dd5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:29.823392 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:30.028260 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44da1e0d3ed044d28cd897ab77724dd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:30.028853 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44da1e0d3ed044d28cd897ab77724dd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:30.036569 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44da1e0d3ed044d28cd897ab77724dd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:30.037316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44da1e0d3ed044d28cd897ab77724dd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:30.050898 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44da1e0d3ed044d28cd897ab77724dd5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:30.053681 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:30.164130 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1305/2595] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:29 2022] DELETE /identity/v3/users/44da1e0d3ed044d28cd897ab77724dd5 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:30.169022 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:30.169688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:30.169688 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:30.190570 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:30.191828 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:30.253294 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1291/2596] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:30.264077 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:30.265530 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:30.324487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b29b-2c17-4e11-9340-43156e3b7a1d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:30.329674 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:30.330291 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:30.330571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:30.330846 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:30.340571 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:30.342015 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0aba5b-d034-450d-9c5d-3a0647702d81 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:30.352071 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1306/2597] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:30.884903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:30.886675 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:30.900604 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7e29-385e-410a-84f7-6f40219ebee2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:30.906011 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:30.906691 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:30.906975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:30.907252 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] PATH_INFO: `/v3/projects/3122b54ae58344d6bfc6c2bb1ae0f981` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:30.908525 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3122b54ae58344d6bfc6c2bb1ae0f981)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:30.910538 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.152697 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3122b54ae58344d6bfc6c2bb1ae0f981', 'request_id': 'req-e3ace244-5f91-41dd-a31b-62af5e51eb9e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:31.160193 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:31.162136 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3122b54ae58344d6bfc6c2bb1ae0f981', 'request_id': 'req-e3ace244-5f91-41dd-a31b-62af5e51eb9e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:31.174507 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3122b54ae58344d6bfc6c2bb1ae0f981', 'request_id': 'req-e3ace244-5f91-41dd-a31b-62af5e51eb9e'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:31.299297 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] The token cache is being invalidate because project 3122b54ae58344d6bfc6c2bb1ae0f981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:31.299605 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:31.300286 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-e3ace244-5f91-41dd-a31b-62af5e51eb9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:31.301827 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1292/2598] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:30 2022] DELETE /identity/v3/projects/3122b54ae58344d6bfc6c2bb1ae0f981 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:31.306877 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.307427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.307427 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.329528 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:31.330770 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:31.394529 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1307/2599] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:31.403139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.403431 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.403760 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.424139 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:31.425362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:31.433467 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1293/2600] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:31.438149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.449333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04048-21d9-4bc9-a90d-871c29149e69 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.454277 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.454894 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.455175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.455449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.456149 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:21:31.456454 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.458056 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.472384 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1308/2601] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:21:31.476975 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.486965 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b092-b8b4-4e2b-96c1-4aa04f32f0e2 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.491770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.492384 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.492664 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.492936 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.493620 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.495514 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.581234 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1294/2602] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:21:31 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:31.585701 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.597138 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b622307-a3c7-4be9-95c1-336fbafc47bb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.660276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.660276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.660276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.660276 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.661105 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.661342 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.878542 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1309/2603] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:21:31 2022] POST /identity/v3/users => generated 295 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:31.882577 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.893751 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7976e-24f1-470c-926d-ea86593613dc admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.898468 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.899073 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.899362 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.899676 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.900321 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.901931 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.912334 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1295/2604] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:31.916325 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.926223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6e185-f465-42d7-9723-e7c0250654b0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.931288 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.931929 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.932218 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.932498 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.933158 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.934787 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdc831-19f6-496b-8a15-1d2b16e912ce admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:31.945369 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1310/2605] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:31.950172 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:31.959963 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c7ab7-816f-4db0-93f2-342a4401abc6 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:31.964407 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:31.965046 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:31.965325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:31.965600 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] PATH_INFO: `/v3/projects/1dc39fe9965749d5b9a83af21eba05b1/users/6d99a31f8a8b49568b4cf4a977863700/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:31.980767 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc39fe9965749d5b9a83af21eba05b1, user_id=6d99a31f8a8b49568b4cf4a977863700, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:31.983415 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:32.093877 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1296/2606] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:21:31 2022] PUT /identity/v3/projects/1dc39fe9965749d5b9a83af21eba05b1/users/6d99a31f8a8b49568b4cf4a977863700/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:32.109849 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:32.110485 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:32.110811 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:32.118047 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:32.143099 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] MFA Rules not processed for user `6d99a31f8a8b49568b4cf4a977863700`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:32.144363 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:32.217143 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1311/2607] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:32.228938 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:32.230666 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:32.288439 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68c006-8a63-4c58-ab5c-c5723ca9cf05 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:32.292606 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:32.293234 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:32.293516 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:32.293790 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:32.301900 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:32.303325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceacc6-d833-48f4-bdcb-6c2825e39b28 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:32.312827 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1297/2608] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:42.308746 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:42.420316 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=31 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:44.101374 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:44.165020 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2384d7e-c8de-4586-8a39-890f35ce0031 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:44.170407 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:44.171168 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:44.171460 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:44.171779 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] PATH_INFO: `/v3/users/6d99a31f8a8b49568b4cf4a977863700` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:44.172933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d99a31f8a8b49568b4cf4a977863700)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:44.174933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:44.336781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d99a31f8a8b49568b4cf4a977863700'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:44.337449 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d99a31f8a8b49568b4cf4a977863700'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:44.345340 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d99a31f8a8b49568b4cf4a977863700'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:44.346028 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d99a31f8a8b49568b4cf4a977863700'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:44.359678 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d99a31f8a8b49568b4cf4a977863700'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:44.437669 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1312/2609] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:44 2022] DELETE /identity/v3/users/6d99a31f8a8b49568b4cf4a977863700 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:44.442351 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:44.442903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:44.442903 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:44.463273 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:44.465158 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:44.543767 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1298/2610] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:21:44.554239 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:44.555728 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:44.620658 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1ff74-4593-4214-a8ab-3d45064e54af admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:44.624514 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:44.625145 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:44.625434 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:44.625713 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:44.637583 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:44.639875 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3ae14-7450-4b9e-8c36-ae358db1ff7a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:44.657846 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1313/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:45.159360 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:45.161272 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:45.179898 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33dbe-18a4-4abd-a0d6-fb98c4f920ac None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:45.188700 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:45.189787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.190278 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.190766 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] PATH_INFO: `/v3/projects/1dc39fe9965749d5b9a83af21eba05b1` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.192906 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc39fe9965749d5b9a83af21eba05b1)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:45.195368 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:45.325831 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc39fe9965749d5b9a83af21eba05b1', 'request_id': 'req-442733b3-0f39-451c-a7f9-83b08139eaa2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:45.333160 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc39fe9965749d5b9a83af21eba05b1', 'request_id': 'req-442733b3-0f39-451c-a7f9-83b08139eaa2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:45.341673 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc39fe9965749d5b9a83af21eba05b1', 'request_id': 'req-442733b3-0f39-451c-a7f9-83b08139eaa2'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:45.443047 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] The token cache is being invalidate because project 1dc39fe9965749d5b9a83af21eba05b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:45.444000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:45.444175 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-442733b3-0f39-451c-a7f9-83b08139eaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:45.445927 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1299/2612] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:45 2022] DELETE /identity/v3/projects/1dc39fe9965749d5b9a83af21eba05b1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:45.452537 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.453148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.453148 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.475100 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:45.476430 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:45.497217 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:45.544867 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1314/2613] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:21:45.554481 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.554770 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.555059 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.577265 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:45.578511 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:45.586848 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1300/2614] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:45.591866 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:45.608386 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8addb3-b03e-46f1-b96f-8fc962e32fbc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:45.613534 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:45.614175 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.614458 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.614736 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.615424 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:21:45.615768 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:45.617561 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:45.631451 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1315/2615] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Oct 30 01:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:21:45.636369 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:45.646199 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ed96b-8706-4104-b895-d30c9e6c7b01 None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:45.651464 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:45.652124 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.652405 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.652679 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.653325 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:45.655263 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:45.775876 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1301/2616] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Oct 30 01:21:45 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:45.780332 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:45.791223 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a94276-8e4f-479c-82e3-dcc05d95afc0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:45.854161 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:45.855330 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:45.855628 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:45.855915 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:45.856555 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:45.858510 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:46.013277 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1316/2617] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Oct 30 01:21:45 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:21:46.017343 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:46.028705 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50edc-e5e4-4a3c-9ac4-990539931007 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:46.033573 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:46.034194 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:46.034487 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:46.034765 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:46.035426 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:46.037078 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:46.048415 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1302/2618] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:46.056924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:46.067860 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ef75a-9cbf-4e7e-a876-8bded71df964 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:46.072789 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:46.073446 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:46.073727 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:46.074006 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:46.074675 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:46.076333 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378457-d95e-4a0f-b7ce-20db9a748461 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:46.088543 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1317/2619] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Oct 30 01:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:21:46.097479 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:46.108810 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174df6-412f-449a-8211-29b40cb14930 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:46.114053 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:46.114787 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:46.115134 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:46.115417 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] PATH_INFO: `/v3/projects/863ed2e08a2d432faaf2bcf2d6944ead/users/c10ef5c7740c47d08053172bb9b6217b/roles/670880c95def468fa84d3961a35799aa` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:46.132389 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=863ed2e08a2d432faaf2bcf2d6944ead, user_id=c10ef5c7740c47d08053172bb9b6217b, role_id=670880c95def468fa84d3961a35799aa)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:46.138001 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:46.272089 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1303/2620] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Oct 30 01:21:46 2022] PUT /identity/v3/projects/863ed2e08a2d432faaf2bcf2d6944ead/users/c10ef5c7740c47d08053172bb9b6217b/roles/670880c95def468fa84d3961a35799aa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:46.283889 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:46.284101 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:46.284359 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:46.313201 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] MFA Rules not processed for user `c10ef5c7740c47d08053172bb9b6217b`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:46.314442 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:46.384508 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1318/2621] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:46.397721 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:46.399715 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:46.425244 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:46.469494 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e343862-b3a7-4c45-a65e-6f866792f2f7 admin admin] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:46.473935 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de8280cded549c58eeabd4e4dc37964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:46.474552 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:46.474833 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:46.475103 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:46.484627 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:46.486077 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0bd2-b952-43d1-8e57-ed29e3c352e3 service cinder] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:46.496013 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1304/2622] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:52.934200 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:52.958865 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:21:53.004590 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3c2d-3d4a-4f7d-8ae7-2e3b137406d2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:53.009838 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:53.010487 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:53.010774 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:53.011053 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] PATH_INFO: `/v3/users/c10ef5c7740c47d08053172bb9b6217b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:53.012268 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10ef5c7740c47d08053172bb9b6217b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:53.014765 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:53.194570 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10ef5c7740c47d08053172bb9b6217b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:53.195152 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10ef5c7740c47d08053172bb9b6217b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:53.202282 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10ef5c7740c47d08053172bb9b6217b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:53.202924 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10ef5c7740c47d08053172bb9b6217b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:53.214933 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10ef5c7740c47d08053172bb9b6217b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:53.306223 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1319/2623] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Oct 30 01:21:52 2022] DELETE /identity/v3/users/c10ef5c7740c47d08053172bb9b6217b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:21:53.311277 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] REQUEST_METHOD: `POST` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:53.311890 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:53.312000 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:53.333920 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.auth.core [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] MFA Rules not processed for user `e23837cd4788413c9fb97c7f11ede732`. Rule list: `[]` (Enabled: `True`). {{(pid=81636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:21:53.335174 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:53.399481 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1305/2624] 10.222.0.26 () {58 vars in 958 bytes} [Sun Oct 30 01:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:21:53.410267 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:53.411759 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:53.480369 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2c961-5aeb-457d-8006-4ed9a7a153d8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:53.484532 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6194924f3054c9598c3548d8e87b294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:53.485155 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:53.485435 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:53.485707 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:53.495380 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:53.496844 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:53.506389 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1320/2625] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:21:53.941644 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] Authenticating user token {{(pid=81636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:21:53.943088 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.fernet_utils [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:21:53.955764 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69de7a-4baf-4b0d-8b52-e6dd0621a34e None None] Validating token access rules against request {{(pid=81636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:21:53.960845 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e23837cd4788413c9fb97c7f11ede732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6345a90487264af499b3342614a87ec3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:21:53.961501 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:21:53.961850 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:21:53.962061 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] PATH_INFO: `/v3/projects/863ed2e08a2d432faaf2bcf2d6944ead` {{(pid=81636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:21:53.963229 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] RBAC: Authorizing `identity:delete_project(project_id=863ed2e08a2d432faaf2bcf2d6944ead)` {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:21:53.965275 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] RBAC: Authorization granted {{(pid=81636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:21:54.093118 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863ed2e08a2d432faaf2bcf2d6944ead', 'request_id': 'req-02c7a0f7-bfef-47e9-8b93-3b799110533c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:54.099060 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863ed2e08a2d432faaf2bcf2d6944ead', 'request_id': 'req-02c7a0f7-bfef-47e9-8b93-3b799110533c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:54.106156 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863ed2e08a2d432faaf2bcf2d6944ead', 'request_id': 'req-02c7a0f7-bfef-47e9-8b93-3b799110533c'} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:54.213521 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] The token cache is being invalidate because project 863ed2e08a2d432faaf2bcf2d6944ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:21:54.213843 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:54.214505 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG keystone.notifications [None req-02c7a0f7-bfef-47e9-8b93-3b799110533c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:21:54.216061 dv-cinder-master-3-smb devstack@keystone.service[81636]: [pid: 81636|app: 0|req: 1306/2626] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Oct 30 01:21:53 2022] DELETE /identity/v3/projects/863ed2e08a2d432faaf2bcf2d6944ead => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:22:03.502072 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:22:04.210207 dv-cinder-master-3-smb devstack@keystone.service[81636]: DEBUG dbcounter [-] [81636] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:23:21.643832 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:23:21.645167 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:23:21.716677 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa5ebe-f81d-4c4f-9d5f-9fcfa319dbd5 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:23:21.777883 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0133918a941b4c3ba07842d5b4deabbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97a13536393943d4b1e22deb299678b1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:23:21.778504 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:23:21.778781 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:23:21.779110 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:23:21.781045 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:23:21.793531 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:23:21.795761 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff82df0-566c-4a7d-aef5-4d974afe1c06 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:23:21.806786 dv-cinder-master-3-smb devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1321/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Oct 30 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:23:31.802092 dv-cinder-master-3-smb devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=58 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}